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Abstract 



The Foster-Lyapunov theorem and its variants serve as the primary tools for studying the stability 
of queueing systems. In addition, it is well known that setting the drift of the Lyapunov function equal 
to zero in steady-state provides bounds on the expected queue lengths. However, such bounds are often 
Oh! very loose due to the fact that they fail to capture resource pooling effects. The main contribution of this 

paper is to show that the approach of "setting the drift of a Lyapunov function equal to zero" can be used 
to obtain bounds on the steady-state queue lengths which are tight in the heavy-traffic limit. The key is 
to establish an appropriate notion of state-space collapse in terms of steady-state moments of weighted 
queue length differences, and use this state-space collapse result when setting the Lyapunov drift equal 
to zero. As an application of the methodology, we prove the steady-state equivalent of the heavy-traffic 
optimality result of Stolyar for wireless networks operating under the MaxWeight scheduling policy. 
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£sj ■ 1 Introduction 

m ' 

The performance of control policies in queueing systems is evaluated by studying the sum of appropriately 
t^J- \ weighted queue lengths, either in steady-state and along almost every sample path. However, deriving op- 
timal control policies is difficult because any stochastic optimal control formulation of the problem is often 
intractable. An alternative is to study the system in heavy-traffic, i.e., when the vector of exogenous arrival 
rates to the queueing system is close to the capacity region in the network. In such regimes, the behavior of 
the network often simplifies: a multi-dimensional state description of the queueing system reduces to a single 
dimension (or to a small number of dimensions) and it is easier to reason about the optimality of the control 
policies in one dimension. This behavior of the queueing system in heavy-traffic, called state-space collapse, 
is at the heart of most heavy-traffic optimality results. 

Heavy-traffic analysis of queueing systems using diffusion limits was initiated by Kingman in |21] and state- 
space collapse was observed for priority queues by Whitt in [32] • The use of state-space collapse to study 
heavy-traffic optimality was introduced by Foschini and Salz in [8] in their classic paper on join-the-shortest 
queue (JSQ) routing. Since then, the methodology and applicability of this technique have been extended in 
a number of papers; see, for example, the works of Reiman [32] , Bramson [5] , Williams [43] , Harrison [16] , 
Harrison and Lopez [T7], and Bell and Williams [2J. This list of papers is by no means exhaustive, it is only 
meant to be a representative sampling of the papers in the area. Many of these papers which consider multi- 
queue models served by multiple resources rely on the so-called resource pooling condition, under which the 
behavior of the queueing system under study in heavy-traffic is governed by a single bottleneck resource. This 
results in the state-space collapse mentioned earlier, which is critical to establishing heavy-traffic optimality. 
In addition, these papers also implicitly assume that the scheduling policy in the queueing system is work 
conserving, i.e., backlogged work is served at the maximum possible rate by each station. In a seminal 
paper on generalized switches, Stolyar extended the notion of state-space collapse and resource pooling to 
systems where such per-node work-conserving policies are hard to define 36 . In particular, he showed that a 
class of scheduling policies called the MaxWeight policies, introduced by Tassiulas and Ephremides [38] (see 
[S7J 02] US] for extensions) is heavy-traffic optimal in an appropriate sense. While Stolyar's work considered 
single-hop traffic only, the proof of heavy-traffic optimality in the multi-hop case was provided by Dai and Lin 
[BJ. Extensions to other types of scheduling policies were presented in [35] by Shakkottai, Stolyar and Srikant. 
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It should be noted that the Max Weight policy was shown to be optimal at all traffic loads for a simple wireless 
network model (with symmetric Bernoulli arrival and service processes) by Tassiulas and Ephremides [39], 
who also obtained optimal policies (which are not of MaxWeight type) for wireless networks where the links 
are arranged in a line |40j . 

In general, state-space collapse does not lead to a one-dimensional state space. When the state-space 
collapse is to a multi-dimensional state, it is often harder to prove optimality in heavy-traffic. A model of the 
Internet with multi-dimensional state-space collapse has been considered by Kang, Kelly, Lee and Williams 
|19j . While the resource allocation policy considered there is not optimal in heavy-traffic, an important 
contribution there is to show that the expected workload is only a function of the number of resources in the 
system and not the number of flows in the system. State-space collapse is key to establishing such a result. 
Multi-dimensional state-space collapse has also been studied by Shah and Wischik for generalized switches 
|33j . In addition to recovering many of the earlier results for other models as special cases, a key contribution 
in [3 3) is the introduction and study of appropriate notions of optimality when the arrival rates lie outside 
the capacity region of the system. Multi-dimensional state-space collapse for a very simple four-link wireless 
network has been considered by Ji, Athanasopoulou and Srikant in |18j who derive the heavy-traffic optimal 
policy for a network which has two bottleneck resources in the heavy-traffic limit. In contrast to heavy-traffic 
limits, Venkatramanan and Lin have shown the optimality of the MaxWeight policies in a large-deviations 
sense [41] . 

Much of work on heavy-traffic analysis of queueing systems relies on showing that a scaled version of the 
queue lengths in the system converges to a regulated Brownian motion. The typical result shows sample-path 
optimality in scaled time over a finite time interval. Often, the results allow a straightforward conjecture 
regarding the distribution in steady-state. Proving convergence to the steady-state distribution is a difficult, 
but not impossible, additional step which is not often undertaken. Some exceptions are the works by Gamarnik 
and Zeevi [9], the earlier mentioned work of Kang et al. [19] . and the recent work of Stolyar and Yudovina 
[37] . In parallel with the development of analyzing the heavy-traffic limits of queueing systems, Harrison [15] 
suggested directly approximating the stochastic arrival and service processes in queueing systems by Brownian 
motions. This theme was further developed by Laws [2H], and Kelly and Laws [3D]. The main idea in these 
papers is to study convex optimization problems suggested by flow conservation equations, along with the 
Brownian control problems. This theme has been influential in much of the work on heavy-traffic optimality 
of control policies for queueing systems. 

Instead of establishing optimality of control policies, if one is simply interested in evaluating the perfor- 
mance of a particular policy, a common technique is to study the drift of an appropriate Lyapunov function 
in steady-state. Assuming that appropriate moments exist, which sometimes might be non-trivial to establish 
as in the work of Glynn and Zeevi [12], setting the drift equal to zero in steady-state immediately provides 
bounds on the steady-state moments of queue lengths. An early use of this technique was used by Kingman 
to derive his well-known bound on the expected waiting time in a G/G/l queue [22] , This idea was pursued 
successfully by Kumar and Kumar [24] and by Bertsimas, Paschalidis and Tsitsiklis )4| who present many 
extensions of the basic idea to different types of queueing systems. The method was extended to loss models 
by Kumar, Srikant and Kumar in [25] . The study of Lyapunov drift to analyze performance has its roots 
in Markov chain stability theory using the Foster-Lyapunov theorem (see the books by Asmussen [T], Meyn 
[28] . and Meyn and Tweedie [30)). An explicit connection between moment bounds and Lyapunov drift-based 
stability was provided by Kumar and Meyn in [23j . While most of these papers obtain bounds on polynomial 
moments, Hajek [14] and Bertsimas, Gamarnik and Tsitsiklis 3 obtained exponential-type bounds on the 
queue lengths. The bounds obtained by Hajek will be very useful to us in this paper. 

Now that we have described prior work on heavy-traffic analysis and performance evaluation of queueing 
systems, we present our motivation for this paper. The Lyapunov drift-based moment bounding techniques are 
simple to derive since they require elementary probabilistic tools. Although the arrival and service processes 
are assumed to be quite simple to apply these techniques, the queueing models and control techniques can 
be quite complicated, thus making the techniques fairly general in their applicability. However, in many 
cases, the bounds obtained from these techniques are extremely loose in complex queueing systems such as 
wireless networks operating under the MaxWeight policy, or sometimes even in simple systems such as parallel 
servers to which arrivals are routed according to the JSQ policy. Therefore, the bounds obtained from drift 
considerations are sometimes not very useful to evaluate control policies for queueing systems. The main 
reason is that, simple drift-based bounds do not exploit resource pooling effects observed in heavy-traffic. 
The key to introducing resource pooling effects into the drift-based arguments is to define an appropriate 
notion of state-space collapse that can be easily incorporated into the derivation of the drift-based moment 
bounds. In this paper, we present techniques for doing so, i.e., we present techniques for introducing the 
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notion of state-space collapse into the drift-based moment bound derivations. In particular, we obtain upper 
bounds on the expected value of weighted functions of queue lengths for queueing systems operating under 
certain control policies, which coincide with lower bounds in heavy-traffic. Additionally, our techniques 
provide explicit bounds on moments even when the system is not in heavy-traffic, and can thus serve as a 
performance evaluation tool for the pre-limit systems when combined with optimization techniques such as 
those suggested in [53] . 

We consider two types of queueing systems to illustrate our methodology: parallel servers where arrivals 
are routed according to the JSQ policy and wireless networks operating under the MaxWeight policy. In 
this paper, we call these two problems, the routing problem and the scheduling problem, respectively. The 
scheduling problem is directly motivated by scheduling in wireless networks and high-speed networks, while 
the routing problem is an abstraction of multi-path routing in communication networks, wireless or wireline. 
The proofs of results are simpler for the JSQ problem, and so for ease of exposition, we illustrate all the steps 
in our derivation using JSQ first. The MaxWeight policy is harder to analyze, but once the basic idea behind 
the proof is presented for JSQ, the extension to MaxWeight follows when the geometrical insight in Stolyar's 
work |36j is translated into the Lyapunov drift framework. We note that the work of Gans and van Ryzin 
[IUJ[TT] is similar in spirit to our work. They study heavy-traffic optimality in terms of steady-state moments. 
They first obtain a lower bound on the workload in a G/G/l queue, which is similar to the Kingman-type 
bound |22] that we also use. However, beyond these similarities, their work is quite different in the following 
aspects: their upper bounds are derived for policies which make explicit use of the knowledge of the arrival 
rates and the topology of the capacity region (which is unknown in wireless networks with time-varying 
channels as we will see later). In contrast, the policies that we study do not require any of this information, 
and further, our Lyapunov drift-based approach appears to be quite different from the techniques used in 
[PUl ITT] . Also related is the work of Gupta and Shroff [13] , who use Lyapunov-based bounds to numerically 
study the performance of MaxWeight algorithms. However, their bounds are not provably tight although 
they perform well in numerical studies and simulations. Thus, they cannot be used to establish heavy-traffic 
optimality. 

1.1 Outline of the Methodology 

The main contribution of the paper is a Lyapunov-drift based approach to obtaining bounds on steady-state 
queue lengths that are tight in heavy traffic. Our approach consists of three major steps: 

1. Lower bound: First, we present a lower bound for the expected queue lengths in a single-server queue, 
and use this to identify appropriate lower bounds for both the parallel server model and the wireless 
network model. The lower bound that we obtain for the single-server queue is perhaps well-known and 
it follows the basic idea behind the Kingman bound |22] . However, we have not seen it explicitly stated, 
so we provide it here along with a short derivation. 

2. State-space collapse: The next step is to show state-space collapse to a single dimension for both models. 
Unlike fluid limit proofs, in our model, the state does not actually collapse to a single dimension. What 
we show is that compared to the queue lengths, the differences between appropriately weighted queue 
lengths is small in an expected sense in steady-state. 

3. Upper bound: The final step is to derive an upper bound on the expected steady-state queue lengths. 
For this step, as we will see, we use a natural Lyapunov function suggested by the resource pooling to be 
expected in the two problems. However, setting the drift of the Lyapunov function equal to zero directly 
does not yield a good upper bound. In addition, one has to use the state-space collapse result from the 
previous step to get an asymptotically tight upper bound. Once the upper bound is derived, checking 
that the upper and lower bounds coincide in heavy-traffic is straightforward. Even when the system is 
not in heavy-traffic, the upper bounds hold and can be of independent interest for performance analysis. 

2 Notation, System Models and Other Preliminaries 

In this section, we introduce some notation that will be used throughout the paper. We consider the control 
of a network of L queues that synchronously evolve in a time-slotted fashion. The evolution of the length of 
queue I is governed by: 



Qi[* + i] 



(Qi[t] + Mt]-Si[W 

Ql[t]+Ai[t\-Si[t\ + Ui{t], for each I = !,••• ,L, 



(1) 
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where Ai[t] and Si[t] respectively denote the amount of arrivals and offered services to queue I in slot t, and 
Ui[t] = max(0, Si[t] — Ai[t] — Qi[i\) denotes the unused service by queue I in slot t. We assume that Ai[t] and 
Si[t] are non- negative integer valued so that Qj[t], for each I, evolves over the space of non- negative integers. 
For convenience, we will sometimes use boldface letters Q, A, S, and U to represent the L-dimensional vectors 
of queue-lengths, arrivals, offered services, and unused services, respectively. In our models in this paper, the 
queue length vectors will form a Markov chain. We will say that the queueing system is stable if this Markov 
chain is positive recurrent. 

Since the queue- length vector Q[i] evolves over integer values in the nonnegative quadrant of the L- 
dimensional real vector space R;V , we will have occasion to use the following notation in the paper and so we 
state them once for convenience: for two vectors x = (xi) and y = (yi) in R L , their inner product, Euclidean 
norm, and the angle between them are respectively given by 



1=1 



\ 



D 

1=1 



= arccos 



(2) 



We use to denote component-wise comparison of two vectors, and and 1 to denote all-zero and 

all-one vectors, respectively. We call two vectors orthogonal, denoted x _L y, if their inner product is zero, in 
which case the Pythagorean Theorem applies: 



-y|| 2 = 



for x _L y. 



(3) 



Additional characteristics and constraints on the arrival and service processes depend on the particular 
problem we will consider, which will be discussed in Sections 12.21 and 12.31 in the context of routing and 
scheduling, respectively. Next, we present a result that provides a bound on the steady-state moment- 
generating function of a random process using drift conditions. 



2.1 A Useful Result 

Our heavy-traffic analysis of the above models relies on a result developed by Hajek [M] in a more general 
context, which will be repeatedly used in the rest of the paper. In particular, this result is useful in proving 
state collapse in the sense mentioned in Section [1.11 We present it below for easy reference. 

Lemma 1. For an irreducible and aperiodic Markov Chain {X[t]} t >o over a countable state space X, suppose 
Z : X — > K + is a nonnegative-valued Lyapunov junction. We define the drift of Z at X as 

AZ(X) 4 [Z{X[t + 1]) - Z(X[t})] l(X[t] = X), 

where X{.) is the indicator function. Thus, AZ(X) is a random variable that measures the amount of change 
in the value of Z in one step, starting from state X. This drift is assumed to satisfy the following conditions: 

(CI) There exists an rj > 0, and a K < oo such that 

E[AZ{X)\X[t] =X]< -r], for all X £ X with Z(X) > k. 

(C2) There exists a D < oo such that 

P(|AZ(X)| < D) = 1, for all X eX. 

Then, there exists a 9* > and a C* < oo such that 



lim sup 1 



< C*. 



If we further assume that the Markov Chain {X[t]} t is positive recurrent, then Z(X[t]) converges in distribution 
to a random variable Z for which 



E 



< C*, 



which directly implies that all moments of Z exist and are finite. 

We now introduce the two queueing control problems that will be considered in this paper. 
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Figure 1: System model for the routing problem. 



2.2 Routing Problem and the JSQ Routing Policy 

In the routing problem, we consider a system of L parallel servers and their associated queues. Packets are 
routed to one of the queues upon arrival (see Figure [lj . The goal is to find a routing policy which minimizes 
the total workload in the system. 

We let A^[t] denote the random number of exogenous arrivals to the router at the beginning of slot t. The 
router is then required to distribute the incoming packets to L queues for service. In other words, in each time 

L 

slot t, the router is expected to select a nonnegative-valued vector A[t] = (Ai[t])i such that A;[t] = As[i]. 

1=1 

Then, the queues will evolve as in ([1]) based on this routing decision. We make the following assumptions on 
the arrival and service processes. 

Assumption 1 (Assumptions for the Routing Problem). We assume that the exogenous arrival process and 
the service processes for different queues are independent (not necessarily identically distributed) . Also, we 
assume that the exogenous arrival process {A^[t]} t and each service process {Si[t]} t is composed of a sequence 
of independent and identically distributed (i.i.d.) nonnegative-integer-valued and bounded random variables 
with As[t] < A max < oo and Si[t] < S max < co, for all I and all t. 

We additionally use the following notation for the mean and variance of different random variables in the 
routing problem: for the arrival process, As = ELAs[l]], erf = war (As [1]); for the service process of each queue 
I, fii = E[5i[l]], v 2 = var(S[[l]); and for the hypothetical total service process, defined as Ss[i] = Ylf=i 
we let /zs — ~Yla=i W) ^s — T^f—i v i ■ Finally, we also introduce the boldface notation for the L-dimensional 
vector of mean and variances for the service processes: fi = (fJ,i)i, and i> 2 = {v 2 )i- Without loss of generalitjQ, 
we assume that fJ, m i n = min; fii > 0. 

We are interested in the performance of a well-known and very natural routing policy, called Join the 
Shortest Queue (JSQ) Routing Policy, defined next. 

Definition 1 (JSQ Routing Policy). For the routing problem introduced above (see Figure\T(), in each time 
slot t, the Join-Shortest-Queue (JSQ) Routing Policy forwards all incoming packets to one of the the queues 
with the shortest queue-length in that time-slot, breaking ties uniformly at random, i.e., given Q[t], and As[t], 
the arrival vector A\t] is selected as 

A[t] = RAND < argmin ( A, Q[t] ) > , 
{{Ayo-.j:, A^A^lt]} J 

where RAND denotes that ties are broken uniformly at random. 

The principle behind the JSQ routing policy is very intuitive: it constantly tries to equalize the queue- 
lengths at all servers by routing all the incoming packets in each time slot to one of the smallest queues. Thus, 
it hopes to make sure that no server is idle when there is work to be done. If this can be achieved, then the 

x We can eliminate any server with = from the system since Si[t] = for all t with probability one for any such server. 
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system will behave as though all the servers pool their resources together and act like a single server queue. 
In this paper, we make this claim precise by using a Lyapunov-based analysis. 

It is clear that the maximum achievable service rate for the parallel queue system is ^ — S;=i /•*!• 
A control policy is said to be throughput optimal if it can stabilize any set of arrival rates which can be 
stabilized by another policy. The JSQ routing policy is well known to be throughput-optimal; additionally, 
the following lemma states that all moments are finite in steady-state. The proof is provided in Appendix lAl 
for completeness. 

Lemma 2. If the mean exogenous arrival rate As lies outside the capacity region 1Z, i.e., As > fis, then the 
queueing network cannot be stabilized by any feasible routing policy. 

For any As in the interior of 1Z, i.e., As < /^s, the JSQ Routing Policy stabilizes the network in the 
following strong sense: {Q[i]}t converges in distribution to a random vector Q whose all moments are bounded, 
i.e., there exist constants {.M r } r= i 2,— such that E[||Q ||'] = M r . 

2.3 Scheduling Problem and the MW Scheduling Policy 

In the scheduling problem, the goal is to select an instantaneous service rate vector for L queues with inde- 
pendent packet arrival processes, subject to certain feasibility constraints on the rates at which the queues 
can be simultaneously served (see Figure [2]). 




Figure 2: System model for the scheduling problem. 



Here, the feasibility constraints can be used to model various types of coupling between service avail- 
ability at different queues. For example, in the context of wireless networks with collision-based interference 
constraints, the feasibility constraints can capture interference among simultaneous wireless transmissions. 
Generically, we let S denote the set of feasible service rate vectors that the controller is allowed to select from. 
Hence, each element S = (Si)i of S is a vector of service rates that can be offered to the queues in one slot, 
and hence it contains the all zero vector 0, corresponding to no service. 

In each time slot t, the scheduler is required to select a feasible service vector S[t] € S. Then, the queues 
will evolve according to ([T]) based on this scheduling decision. We make the following assumptions on the 
arrival processes and the set of feasible service rates. 

Assumption 2 (Assumptions for the Scheduling Problem). We assume that the arrival processes to different 
queues are independent (not necessarily identically distributed). Also, we assume that each exogenous arrival 
process {Ai[t]} t is composed of a sequence of i.i.d. nonnegative-integer-valued and bounded random variables 
with Ai[t] < A max < oo for all I and all t. We assume that each feasible service rate vector S € S is 
nonnegative-integer-valued and bounded with Si < S max < oo, for all I. 

We additionally use the following notation for the mean and variance of different processes in the scheduling 
problem: for the exogenous arrival process for each queue I, A/ = EL4/[1]], of = var(Ai[l}); for the service 
process of each queue I, /// = E[S/[1]], vf = var(Si[l]); for the hypothetical total arrival process, defined as 
As[t] = 53 i=1 j4./ [t] , we let As — J2f=i^h °s — J2f=i a f- Finally, we also introduce the boldface notation 
for the L-dimensional vector of mean and variances for the arrival and service processes: A = (A/)/, and 
cr 2 = (of)/; and /x = and u 2 = (vf)i. Without loss of generalitjH, we can assume that A; > 0, V7. 

2 We can eliminate any arrival with A; = from the system since Ai[t] = for all t with probability one for any such arrival. 
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In the rest of paper, we are interested in the performance of a well-known queue-length-based scheduling 
policy, called Maximum Weight (MW) Scheduling Policy, defined next. 

Definition 2 (MW Scheduling Policy). For the scheduling problem introduced above (see Figure^!, in each 
time slot t, the Maximum Weight (MW) Scheduling Policy selects the service rate vector S[t] from S to 
optimize the following objective, breaking ties uniformly at random, 

S [t] = RAND \ arg max( Q [t] , S ) 1 
where (■,■) is the vector inner product in R L defined in 0). 

2.4 Capacity Region for the Scheduling Problem 

Definition 3 (Maximum Achievable Rate (Capacity) Region for the Scheduling Problem, 1Z). For the schedul- 
ing problem described in Section \2. 3\ operating under Assumption\Mand the given set of feasible service rate 
vectors S, the maximum achievable rate region TZ is the convex hula of S, i.e., 

TZ = Convex Hull(S). 

Under the assumed finite size and nonnegative nature of the set S, the region TZ becomes a polyhedron in the 
nonnegative quadrant of R L , and hence can be equivalently described by 

TZ = {y>Q:(c^\y) <b (k \ fc = 1, - - - ,K}, (4) 

where K denotes the finite (and minimal) number of hyperplanes that fully describe the polyhedron, and the 
pair (c^ k ',b^) defines the k th hyperplane, W k ', through its normal vector 6 R L and its inner product 
value b^ E K asH (k) = {r : (c^.r) = M fe )}. Further, since the allowed feasible service rates are nonnegative- 
valued, the pairs (c«,&( fc )) fe , are assumed to satisfy: 

|| c (fe)|| =1) c «>-o, b {k) >0, fork = !,■■■ ,K. 

Furthermore, we call the intersection of the k th hyperplane with 1Z the k th face J 7 ^ of the achievable rate 
region, i.e., T (k) = U {k) n TZ. o 

Figures [3] and |4] illustrate the above concepts for a 2-dimensional and a 3-dimensional capacity region. The 




S: Red points 



Figure 3: The figure illustrates the capacity region 
for a 2-dimensional example, indicating the hyper- 
planes and the associated normal vectors . 



J2 




Figure 4: The figure illustrates the capacity region 
for a 3-dimensional example (L = 3), indicating that 
each is polyhedral in K i_1 with a non-empty 

interior. 



next lemma states the well-known throughput-optimality of MW Scheduling, and shows that the steady-state 
moments of the queue length vector are bounded. As in the routing problem, the proof of the following lemma 
uses Lemma [Hand is omitted. 

3 The convex hull of a set S C is the smallest convex set that contains S. 



7 



Lemma 3. If the arrival rate vector A y lies outside the capacity region 1Z, i.e., X^TZ, then the queueing 
network cannot be stabilized by any feasible scheduling policy. 

For any arrival rate vector A in the interior oflZ, i.e., A € Int(lZ), the MW Scheduling Policy stabilizes 
the network in the following strong sense that the queue-length vector process {Q[£]}t converges in distribution 
to a random vector Q whose all moments are bounded, i.e., there exist finite numbers {M r } r= i 2 ... such that 
E[||Q|n = M r . 



3 Lower Bounds 

We first consider a simple single-server queueing system, depicted in Figure [5] and obtain a lower bound on 
the steady-state expected queue length in this system. This lower bound will then be used to obtain lower 
bounds for both the routing and scheduling problems. 



a[t] 



0[t] 



PM 



Figure 5: Lower bounding system with i.i.d. arrival and service processes over time with bounded support. 

We assume that the arrival and service processes to the single-server queue are, respectively, described by 
two independent sequences of i.i.d. nonnegative-valued random variables {a[i]}t and {/3[t]}t- We assume that 
both distributions have finite support, i.e., there exist a max < oo and (3 max < oo such that a[t] < a max and 
/3[t] < fimax with probability 1 for all t. We also denote the means and variances of the arrival and service 



processes as a = E[a[l]], cr^ = uar(a[l]), and [3 = E 
server $[t] evolves as 



var{[3[\\). Then, the queue-length of the 



$[t+l] = ($[*] + a[t] -P[t]) + , *>0. 
We are now ready to provide a useful lower bound on the steady-state performance of this system. 



(5) 



Lemma 4. For the system of Figure [5| with a given service process {f3[t]} t , consider the arrival process 
{oS e ^[t]}t, parameterized by e > 0, with mean satisfying e = j3 — oM\ and with variance denoted as cr 2 ^ . 
Let the queue-length process, denoted by {3?^[i]}t, evolve as in 0j with a[t] := a {t) [t\. 

Then, {$ (e) [t]} t is a positive Harris recurrent Markov Chain (J30j) for any e > 0, and converges in 

distribution to a random variable $ with all bounded moments. 
Moreover, the mean of $ can be lower-bounded as 



E 



> 



C (e) 

B l , 

2e 



(6) 



where C (e) ^al + uj + e 2 , and B l ± & 



p ' ^ ' """" "1 — 2 

Therefore, in the heavy-traffic limit as the mean arrival rate approaches the mean service rate from 



below, i.e., as e \. 0, and assuming the variance a (e) converges to a constant a^, the lower bounds become 



lim inf eE 



> 



c 

2" 



(7) 



where £ = a\ + v% 

Proof. In the following argument, we will omit the ^ superscript for ease of exposition, and revive it when 
necessary. The claim that {$[f]}t is positive Harris recurrent follows from standard negative drift conditions 
([30]). Then, Lemma [1] applies to the Lyapunov function V($) = ||$|| as in the proofs of Lemmas [2] and El 
and is omitted here. These establish that $[t] converges in distribution to $ with all bounded moments, i.e., 
E[||$|| r ] < oo for each r = 1,2,- ■■ . 

To prove the lower bound ([6]), we first expand ([5]) as 



*[t+l] = $[t] + a[t]-)8[i]+X[*], *>0. 



(8) 
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where x[A denotes the unused service in slot t. For the quadratic Lyapunov function = ||$|| 2 , the mean 

drift AW(<Z>) = [W($[t + 1]) - W($[*])] X($[t] = $) in steady-state must be zero, i.e., E[AW(f)] = 0. Next, 
we expand the conditional mean drift of W , omitting the time reference [t] for brevity: 

E[Aiy($) | = $] = E[($ + a - /3 + x) 2 - $ 2 | 

= E[($ + a-/3) 2 +2($ + a-/3)x + x 2 -$ 2 | *] 
= E[(q - /3) 2 | $] + 2E[(a - /3) | $]$ - E[x 2 | $] 

where the last step uses the fact that \(® + « — /?) = — X 2 by definition, and the independence of the arrival 
and service processes from each other, and from $. Taking expectations of both sides with respect to the 
steady-state distribution of {$[£]} and using the fact that E[AVT(<I>)] = yields, after re-organizing and 
reviving the ^ notation, 

fE ($ (0 ] = 3 a(e) -^ E lx(* (e) ) 2 ] 



2 

C (e) 



> 

- 2e 2 ' 

where the last step follows from simple manipulations of the first term, and from the facts that E[x($ )] = e, 
and x[t] < Pmax for all t. o □ 

Next, we will discuss the implications of this lower bound on the routing and scheduling problems, respec- 
tively. 

3.1 Lower Bounds for the Routing Problem 

To lower bound the performance of any feasible routing policy, we assume resource pooling whereby we 
consider a hypothetical single server that serves all exogenous arrivals with a service rate of {Ss^]} defined 
as the sum of the service processes of all servers in the actual system, i.e., = JZf—i This results in a 
single-server queueing system of Figure [5] with the arrival process a[t] = As[t], for all t with ct max = A max , 
and the service process /3[t] — Ss[t], for all t with (5 ma x = LS max . It is then easy to see that the corresponding 
queue-length process {$[t]}t is stochastically smaller than the total queue-length process {J2i=i Qi[t]}t of the 
original multi-server system in Figure [T] under any feasible routing policy. Thus, utilizing the lower bounds 
from Lemma |4j we next establish lower bounds on the performance of any routing policy (and hence JSQ) 
for all arrival rates. 



Lemma 5. For the routing problem of Section \2.2\ with a given service vector process {S[t]}t, consider the 
exogenous arrival process {A^[t]} t , parameterized by e > 0, with mean A^ satisfying e = /i£ — A^, and with 



variance denoted as (a^) 2 . Accordingly, let the queue-length process under JSQ Routing with this arrival 
)(£ )r,n , -•- ™ 



process be denoted as {Q( c )[i]} t , evolving as in (Qp. Moreover, let Q denote the limiting random vector with 
all bounded moments that the process {Q^[i]}t is known (by Lemma\3j) to converge to, for any e > 0. 

Then, the first moment of can be lower-bounded, with notation = (cr^) 2 + + e 2 , as: 



E 



L 

i=i 



A LS„ 



where B± — 

Therefore, in the heavy-traffic limit as the mean arrival rate approaches the mean service rate from 
below, i.e., as e ! 0, and as the variance (o-^) 2 converges to a constant o~\, the lower bound becomes, with 
notation £ = <r§; + v\, 



lim inf eE 



£<T 

L/=l 



> \- do) 
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3.2 Lower Bounds for the Scheduling Problem 



Unlike the routing problem, where there is a single natural lower bound on the total queue-length in the 
system, in the case of scheduling, the polyhedral nature of the capacity region 1Z (see Definition [3J) , implies 
that there are many possible lower bounds. 

Assume that A € Int(1Z). From Definition [31 recall that the capacity region is bounded by K hyperplanes, 
each hyperplane Tl^ described by its normal vector and the value b^ k \ Then, we can define the distance 
of A to the k th hyperplane and the closest point, respectively, as 

e (fc) = min ||A-r||, (11) 

A M a A + e « c M, (12) 

where e^ k ' is strictly positive for each k since A is assumed to be in the interior of 7Z. We let e = (e^ fc ')|L 1 denote 
the vector of distances to all hyperplanes, which will later be used to alternatively refer to the arrival rate 
vector A. Noting that A^" may be outside the capacity region 1Z for some hyperplanes, we next distinguish 
those for which A*- fc ' is within 1Z as 

£\ = {ke{l,--- ,K}: \ {k) €1Z}. (13) 

Notice that K,^ identifies the set of dominant hyperplanes whose closest point to A is on the boundary of the 
capacity region 1Z, hence is a feasible average rate for service. We note that for any A £ IntiJZ), the set K.^ is 

non-empty, and hence is well-defined. Figure [6] illustrates the distance and the projection of a given 
arrival vector A for the 2-dimensional rate region of Figure [3J 



r 2 




Figure 6: Illustration of the introduced parameters and A^ in the 2-dimensional rate region of Figure [3j 
Note that, for the selected A, K\ includes the first two faces, but not the third face since A*- 3 -* is outside the 
rate region 1Z. 

Associated with each hyperplane, say W k \ we construct the following hypothetical single-server queue: 
following the notation in Figure[S]with the superscript ( k > to distinguish different hyperplanes, we set [t] = 
( c( fc ) , A[t] ) , where A[i] is the vector of arrivals to the L queues in the scheduling problem, and /3 < - fc - ) [t] = b^ k ' , 
where M fe ) is the positive constant defining the value of the k th hyperplane. Accordingly, the single server 
queue-length $( fe )[£] evolves as in ([5]) with these arrival and service process mappings. Then, {^^[i]}* is 
stochastically smaller than the queue-length process {( c^ k \Q[t] )} t under any feasible (and hence MW) 
scheduling policy. Hence, utilizing Lemma SJ we can find the following lower bounds on the moments of the 
limiting queue-length vector under MW Scheduling. 

Lemma 6. For the scheduling problem of Section \2.3\ with a given set of feasible schedules S, consider the 
exogenous arrival vector process {A< e )[i]} t with mean vector A^ 6 -* G Int(lZ) and with variance vector denoted as 
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(er( e )) 2 = ^(c^) 2 ^ ■ Here, the superscript vector is given by e = {e^)^ =1 >- 0, where and gives the 

distance and the projection of to the k th hyperplane as defined in ill]) and U2\) , respectively. Accordingly, 
let the queue-length process under MW Scheduling with this arrival process be denoted as {Q( e '[i]} t , evolving 

as in (QJ). Moreover, let denote a random random vector with the same distribution as the steady-state 

distribution of {Q^ [t]}t (by Lemma\Bj). 

L 

Then, for each k G {!,••• t K}, and with C (e ' fe) = ( (c^) 2 , {a^f ) = V(c ; (fc) ) 2 (CT z (C) ) 2 , 



:«,Q (e) 



> - — 77-T — (14) 
- 2 e<» 1 



where B[ k) 4 ^. 

Further, for each k £ /C^(C), consider the heavy-traffic limit e^ k ' J, 0; and suppose that the variance 
vector (<r( e )) 2 approaches a constant vector cr 2 . Then, defining = ( (c^- 1 ) 2 , er 2 ), we have 

;c«Q (e) )l > ^ (15) 



liminf e (fe) I 



4 State-Space Collapse 

In this section, we will show a moment condition which is the equivalent of state-space collapse in prior 
literature of fluid and diffusion limit analysis of the routing and scheduling problems. Specifically, we will 
show that the steady-state queue-length vector concentrates around a line within the L-dimensional state 
space in the following sense: the deviations from the line are bounded, independent of heavy-traffic parameter 
e. Since the lower bounds on the queue lengths are of the order of 1/e, this would then show that the deviations 
from the line are small compared to the queue lengths themselves. To establish state-space collapse, in both 
problems, we will first identify a vector and show that the queue lengths concentrate along this vector. For 
this purpose, we first introduce some notation. 

Let c y be a vector with unit norm. The components of the queue length vector Q parallel and 
perpendicular to c are, respectively, given by: 

Q|| = (c,Q)c, 

Q± = Q Q = [Qi - { c, Q ) q]f =1 . 

If c represents the direction along which state-space collapse occurs, then we study the statistics of Qj_ to 
show that all its moments are bounded by constants that do not depend on the proximity of the arrival rate 
vector to the boundary of the capacity region. To that end, we utilize Lemma [1] by studying the drift of the 
Lyapunov function Vj^(Q) = ||Q_l|| to show that: (i) the drift of this Lyapunov function is always bounded; 
(ii) and that, when ||Qj_|| value is sufficiently large, it has a strictly negative drift whose value does not depend 
on the distance of the arrival rate vector from the boundary of the capacity region, denoted as e and e in 
Sections 13.11 and [3.21 This latter property will be important in establishing the heavy-traffic optimality of the 
schemes. It turns out that studying of the drift of ||Q_l|| is hard; instead, as we will see later, it is easier to 
study the drift of ||Q|| 2 and ||Q|||| 2 . The following lemma serves as a useful preliminary step which relates 
the drift of these three quantities; the proof is provided in Appendix |Bj We will use this result in deriving 
our state-space collapse results later. 

Lemma 7. Consider the generic queueing system described in Section with queues evolving according to 
rijj, where the arrival and service processes are respectively bounded by A max and S max for each link and each 
time slot. Define the following Lyapunov functions: 

Kl(Q) = HQj.ll, W(Q) ^ IIQH 2 , and VK||(Q)^||Q|||| 2 , (16) 
with the corresponding single-step drifts denoted by: 

AKl(Q) 4 [V ± (Q[t + l})-V ± (Q[t])]X(Q[t] = Q), (17) 

AW(Q) 4 [W(Q[t + 1]) - VF(Q[t])] l(Q[t] = Q), (18) 

AW„(Q) 4 [W,|(Q[t + l])-W||(Q[t])]X(Q[*] = Q). (19) 
Then, the following properties hold for the random variable AVj^(Q): 
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1. We can bound the drift ofV± with the drifts ofW and W\\ as follows: 

AVx(Q) < -j^j (AW(Q) - AW {l (Q)) , for all Q € K+. (20) 

2. The drift of V± is absolutely bounded as: 

\AV±(Q)\ < 2^Lm^{A max ,S max ), for all Q € K+, (21) 

where, recall from Section [H that, A max and S max respectively bounds the number of arrivals to and 
departures from a queue, and L is the number of queues in the network. 

Notice that (|2ip verifies Condition (C2) of Lemma Q] with 

D = 2\ZT, max(A max , S max ) 

for the Lyapunov function Vj_(Q). To conclude that Vj_(Q[i]) converges to ||Qj_|| with finite moments, we also 
need to verify condition (CI) of Lemma [TJ In the following subsections, we do so for both the JSQ Routing 
and MW Scheduling policies. Further, we will show that the moments bounds on ||QjJ| are independent of 
the proximity of the arrival rate vectors to the boundary of the capacity regions - a key feature to be used 
for the proof of heavy-traffic optimality of these policies in Section [5] 



4.1 State-Space Collapse for the Routing Problem under the JSQ Policy 

We recall the model introduced in Lemma [5] for the routing problem, where the exogenous arrival process 
is parameterized by a positive valued e = fj,s—X^\ and where the queue-length process and the limiting 

queue-length random vector under the JSQ Router are respectively denoted by {Q( e )[i]} t and . Recall 
that the JSQ policy attempts to equalize the queue lengths, so one would expect the state-space to collapse 
along the direction of a unit vector, all of whose components are equal, i.e., the vector c = = (-7=)^. 

Then, the projection and the perpendicular of any given Q^ e ^ € with respect to this line become: 



Qf, £) ^ ^1, and QW A 



Qi — j^Q^s 



L 



1 = 1 



where = J2iLi Qi denotes the total number of packets in the network. We already know, from Lemma[2] 
that Q*- 6 - 1 ^] converges to in distribution for any e > 0. Due to continuous mapping theorem, this implies 
that Qy^[i] also converges in distribution to Q|^. The following result establishes the boundedness of all 
moments of the limiting Qm ^ uniformly for all e > 0. 

Proposition 1. Consider the limiting queueing process ^ under JSQ Routing, serving the exogenous 
arrival process {A^ }t parameterized by e = /is — . Suppose we fix some 5 G (0, /i m , n -\/L) and consider 
any e € (0, fi m in"\/L — 8), where fi m i n — min; //j. Then, there exists a sequence of finite numbers {N r } r =i,2,— 
such that E ^ 



< N r , for all e € (0, (j, m i n vL — S) and for each r = 1, 2, 



Proof. The proof follows from Lemma Q] applied to the Lyapunov function Vj_(Q^ e ^) = |]Q^||, where the fol- 
lowing argument omits the superscript ^ for ease of exposition. Note that ([21]) already establishes Condition 
(C2) of LemmaQ] Thus, the proof is done once we verify Condition (CI) for the JSQ policy. To that end, we 
use the bound on AVj_(Q) in (|20|) and study the mean conditional drift of AM^(Q) and AW\\ (Q) next. 

We start by upper-bounding E[AW(Q) | Q] = E[AVK(Q) | Q[t] = Q]. In what follows, we will omit the 
time reference [t] after the first step for brevity. 

E[AW(Q)|Q] = E[||Q[i + l]|| 2 -||Q|| 2 |Q] 

= E[||Q + A-S + U|| 2 -||Q|| 2 |Q] 

= E[||Q + A-S|| 2 + 2(Q + A-S,U) + ||U|| 2 - ||Q|| 2 |Q] 

< E[||Q + A-S|| 2 -||Q|| 2 |Q] 

= E[2(Q,A-S) + ||A-S|| 2 |Q] 

< 2E[(Q,A-S)|Q]+Jfi, (22) 
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where the inequality (a) follows from the fact that Ui(Qi + A\ — Si) = —U? < 0, for each I, and K\ = 
Lmax(A mal , S max ) 2 is bounded since both the arrival and service processes are bounded. 

Next, we bound the first term in (|2"2"|) by first defining a hypothetical arrival rate vector A = (A;); 
with respect to the given service rate vector fi and the e € (0, UminvL ~~ &) as A = — Note that 

A mm = min; A; > S, where S is a fixed constant in (0, /i m m\/L), and that Y]f—-, A; = /is — e, which is identical 
to the assumed rate As of the exogenous arrival process {As[i]}t. The following derivation starts by adding 
and subtracting A to the first term in ([22]) : 

E[(Q,A-S)|Q] = (Q,E[A|Q]-A)-<Q,m-A> 



(a) 
(b) 



< 



(<0 
< 



(d) 
< 



< 



(Q,E[A|Q]-A)-^=<Q,c) 

E[A E |Q] Qmin- <Q, A) - -^|| Q|| 

L 

As Qmin — ^ A;Q; 7f IIQ|| I 

i=l ^ 
L ^ 
— Qmin) TpllQ||l 

L 

Amm ^ ^ | Qmin | 7= 1 1 Q || I 

(=1 VL 



|Q Qmin ^r, 



IQn 



Q Qmin 1|| A m i n ,_||Q|| 



IIQnll 



where 



Or. 



min Q m > 0; 

Km<L 



x||i = is the /i-norm of x, 



i=i 



and step (a) follows from the definition of A; (b) follows from the definitions of the JSQ policy (see Definition!]]) 
and of Qy; (c) is true since l\ norm ||x||i of any vector x g M. L is no smaller than its In (Euclidean) norm 
||x||; (d) is true since \Qt. minimizes the convex function ||Q — y 1|| over i/gl; (e) is true since \ m in > 5 hy 
construction of A. This bound, when substituted in (|2"2"|) , yields: 



E[AW(Q)|Q] < -2<y||Q x ||-^=||Q|| 

Next, we lower bound E[AWj| (Q) | Q] = E[AW\\ (Q) | Q[t] = Q]: 
E[AW|i(Q)|Q] = E 



(23) 



[(c,Q[t + l]) 2 -(c,Q[i]) 2 |Q] 
[(c,Q + A-S + U) 2 -(c,Q) 2 |Q] 
[( c, Q + A - S ) 2 + 2( c, Q + A - S ) ( c, U ) 
(c,U) 2 -(c,Q) 2 |Q] 
[2(c,Q)(c,A-S) + (c,A-S) 2 
+2( c, Q + A - S ) ( c, U ) + ( c, U ) 2 | Q] 
= 2(c,Q)(c,E[A|Q]- M )-2E[(c,S)(c,U) | Q] 

+E [( c, A - S ) 2 + 2( c, Q + A ) ( c, U ) + ( c, U ) 2 | Q] 
> 2(c,Q)(c,E[A|Q]- M )-2E[(c,S)(c,U) |Q], 



= E 



= E 



(24) 
(25) 
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where the inequality follows from the observation that (|M|) >0asc^O, U^O, Q^O. We further lower- 
bound the remaining terms in (|25|) individually. For the first term, we first recall that ( c, Q ) = ||Q|| ||. Also 
noting that c = \j\J~L here and that As = /^s — e, 

<c,E[A|Q]- M ) = -^(E[4|Q]- W ) = = 

Finally, we can lower-bound the second term in (|2"51) easily by — K 2 , where K 2 = 2LS max , since Si < S max 
and Ui < Smax for all Thus, using these lower bounds in (f25j) yields: 



E[AWj,(Q)|Q] > -^||Q|,||-jr 2 . (26) 



We now substitute the bounds (|23|) and (|26|) in (|20|) . and cancel common terms, to bound the conditional 
mean drift of Vj_(Q) as 

E[AV ± (Q)|Q] < + 

where 5, and ^2 ar e positive constants defined above, all independent of e. Note that this inequality verifies 
Condition (CI) of Lemma [TJ and hence establishes the claimed existence of finite constants {iVr}r=i,2,— for 

which E[||Q (e) ||' r ] < N r , for all e G (0,/w, v 7 ! - 6), and for each r = 1,2,- • • . o □ 

This proposition establishes that even for a heavily loaded network, where e approaches zero, the steady- 
state queue-length vector concentrates around the line along c, and the difference between the queue lengths 
have bounded moments. In contrast, the lower bound on the total queue length goes to infinity as e goes 
to zero. Thus, in comparison to the total queue length, the differences in the queue lengths are negligible. 
This observation will be critical in Section [5] for arguing that the queueing network performs close to the 
lower-bounding resource-pooled system of Section [3J Since relative to the queue lengths, the differences in the 
queue lengths are negligible, the queue lengths can be thought as being attracted towards the line defined by 
the vector c. We will call this line, the line of attraction. 

4.2 State-Space Collapse for the Scheduling Problem under the MW Policy 

We recall the model introduced in Lemma [6] for the routing problem, where the exogenous arrival vector 
process {Ag }t is parameterized by the vector e >~ that measures the distance of the arrival rate vector 
G Int(lZ) to each of the K hyperplanes of 7Z. The corresponding queue-length vector process and the 
limiting queue-length random vector under the MW Scheduler are respectively denoted by {Q^[i]}t and 

Q (e) . ' 

Different from the routing case, in the scheduling problem, depending upon the arrival rate vector, the 
line of attraction can be different. In general, associated with each hyperplane enclosing the capacity region 
1Z (cf. Definition [3J is a possible line of attraction. Recall that the pair (c^ , ) defines the k th hyperplane 
%( k ) through its unit norm normal vector c( fe ) and its value b^ k ' > 0. Accordingly, we define the projection 
and the perpendicular vector of any given Q( e ) with respect to the k th normal as: 

QP 4 (c«,Q( e >)c« 
Q (e,fc) a Q (e) _ Q (e,fe)_ 

Also recall the notion of dominant hyperplanes K, , <ej from (|13[) to note that for each k G /C» (e> , the corre- 
sponding projection A^ e ' fc - ) as defined in (TT2"j) lies on the k th face J 7 ^ (see Definition [3]) of the capacity region 
1Z. Next, we further distinguish those faces in ^-\(€) for which A < - e,fc - ) is an interior point on the face, i.e., 
j s no ^ shared by more than one hyperplane: 

K° £ , = {ke JC xie) : A (e ' fc) G Relint(^)}, (27) 

where Relint(J 7 ( k >) indicates the relative interior of the polyhedral set J 7 ^, which is non-empty for each k 

by construction^- It is also easy to verify that, for each A' 6 -* G Int{TZ), the set K° (C) is non-empty. In the 

A 



This follows from the assumed minimality of the set of hyperplanes that define the capacity region 1Z. 
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example of Figure[Sl we can see that fC^ = {1, 2}, while K,\ = {2} since the projection A*- 1 -* is at the corner 

of face J-^\ hence is not in its relative interior. With this terminology, we can now present the state-space 
collapse result for the MW Scheduler. 

Proposition 2. Assume X^ £ Int(JZ) in the scheduling problem with the corresponding set of {e^)} fc as 
defined in ill]) . Then, under the MW Scheduling Policy, for each k G K° (£) , there exist finite constants 

{JVr^}r=l,2,— such that E ||Qj_ || r < Nr k \ for all e >- 0, each k £ /C° (C) , and each r = 1,2, • • • . 



A ( 

Proof. Assuming a given e >- 0, the following argument omits the superscript ^ for notational convenience 
so that Q' c ', X^, Q( e,fe ) are respectively denoted as Q, A, Q( fc ). For any k € }C\, let us recall the Lyapunov 

functions in (1161) to define their counterparts associated with the normal vector c 1 --^ : 

^ fe) (Q) = ||Qi fe) ||, ^(Q)^||Q|| 2 , and W|[ fc) (Q) = ||Q[| fe) || 2 , 

The rest of the proof follows the same line of reasoning as in the proof of Proposition[T]to bound AV^ (Q) by 
utilizing (|2"0|) . However, before we start analyzing the mean conditional drift of AW(QW) and AW y (QW), 
we need to establish some basics related to the geometry of the scheduling capacity region 1Z. 

Since A is in the interior of 1Z, for each k e K.\ , we have A^ fe ' in the relative interior of J 7 ^ by definition. 

This means that, for each k G K.%, there exists a small enough <JW > such that the set 

fijgj '#n{reMf : ||r-AW|| < 

denoting the set of vectors on the hyperplane Ti^ that are within distance from A^, lies strictly within 
the face . 

We are now ready to upper-bound E[AVF(Q) | Q] = E[AVF(Q) | Q[t] = Q] exactly as in the derivation of 
to obtain: 

E[AW(Q)|Q] < 2E[(Q,A-S)|Q]+ifi, (28) 

where K\ = L m&x(A max , S max ) 2 ■ Next, we use the definition of projection X^ from (|12p to expand the first 
term as: 

E[(Q,A-S)|Q] = (Q,AW- e WcW)-E[(Q,S)|Q] 
= -e( fe '(Q,c«) + (Q,AW-E[S|Q]) 

= - e W||Q[ | fc) || + (Q,A (fe) -E[S|Q]) (29) 
We note that the definition of the MW Scheduler (see Definition [2]) directly implies that 

(Q,E[S|Q]> = max(Q,r), (30) 

r£TZ 

where Ti is described in Definition^ Thus, noting that the set M^j C 1Z, we can upper-bound (|29| as follows: 



< 




IQf| fc) ll 


+ min ( Q, A 








5< fc ) 


(a) 


_ e (fe) 


IQ^II 


+ min ( Q , 










(1) 


_ e (fe) 


IQf, fc) ll 


-«5«|| Q f || 



where (a) follows from the facts that = Q — ( c( fe ) , Q ) c( fc ) by definition, and that the vector A*-*' — r 
is perpendicular to the normal vector c' ' since both A (fc) and r are restricted to be on the hyperplane US >\ 
(b) follows from a key idea in |36j where it is noted that the inner product between is minimized when 
r is selected on the boundary of i.e., with ||A^ — r|| = S^ k \ such that A' fe ) — r points in the opposite 

direction to . Substituting the above result in (|28[) yields our first bound: 

E[AW(Q)|Q] < -2eW||Qf | fc) || -2,5W||Q^ ) || (31) 
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The lower-bounding argument on E[AW^[ fe) (Q) | Q] = E[AW^[ fc) (Q) | Q[t] = Q] follows the exact same steps 
until (f24|) as in the JSQ case, with the minor modification in the final expression (f25j) that the conditioning 
on Q remains for the service vector S instead of the arrival vector A, since the MW Scheduler controls S 
while the JSQ Router controls A. This yields 

E[A^[ fc) (Q) | Q] > 2(c( fe \Q)(c«,A-E[S|Q]) - 2E [( c< fc \ S )( c< fc \ U ) |Q 

> 2(c( fe ),Q)( C W,A-E[S|Q]) -K 2 
( = 3 -2 £ W||Q[ | fc) || +2||QW||min(c,AW -r) - K 2 

> -2 £ ( fe )||Q( fc) || + 2||Q< fc >|| ( ( c, A« ) - M fc >) - K 2 



> -2eW||Q[ fe) || -K 2 , (32) 

where step: (a) is true for K 2 — ^TS^^; (b) uses the definitions of projection A^ and Q|| fc ' together with 
the fact that the MW Scheduler achieves ^] (c) is true since {r y : ( c^ k \r ) < b^} D K from (g]); and 
(d) follows from the fact that ( c( fc ), A (fc) ) = &W since A (fc) G U {k) . 

Finally, utilizing the bounds (|31[) and (|32l) in ([20]) . and canceling common terms, yields the following 
bound on the conditional mean drift of vf°(Q) as: 

E[A^ fe) (Q)|Q] < -5^ + Kl + rt f 2 , foreachfce/C ,,, 

2||Qrl| 

where 5^ , K\, and K 2 are positive constants defined above, all independent of e. By verifying Condi- 
tion (CI) of Lemma [TJ this result proves the claimed existence of finite constants {Nr k ^} r =i,2,--- such that 



E 



|Qt fc) l 



< ivi fe) , for all e y 0, each k £ IC° {£} , and each r = 1,2,- •• . □ 

A 



The above proposition provides bounds on the deviation of Q from different lines of attraction correspond- 
ing to the normals of different hyperplanes in K,° (£) . However, in the heavy-traffic limit as A^ 6 -* approaches 

A 

the relative interior of one of the faces, only one of the hyperplanes becomes a dominant face and therefore, 
there will be a single line of attraction associated with the normal of that face only, as one would expect. 



5 Upper Bounds and Heavy- Traffic Optimality 

The purpose of this section is to derive upper-bounds on the steady-state queue-length through simple 
Lyapunov-based methods that will be shown to be asymptotically tight under heavy-traffic, i.e., when the 
arrival rates will approach the boundary of the capacity regions. In particular, we use the state-space collapse 
results established in Section H) 

As in the case of the state-space collapse results, here too the analysis is similar for both the JSQ and 
MW Policies. The derivation of the upper bounds for both cases use the following lemma. 

Lemma 8. Consider the generic queueing system described in Section^ with queues evolving according to 
(QP, where the arrival A[t] and service S[t] vectors at time t are allowed to depend on Q[t]. To indicate 
this dependence, we will use the notation A(Q) and S(Q) to refer to these two processes. Suppose {Q[t]} t 
converges in distribution to a valid random vector Q, with all bounded moments, i.e., E[||Q|| r ] < oo for each 
r = 1, 2, • • • . Then, for any positive vector c £ we have 

WW rw «^ Afnnl E [( c, A(Q) - S(Q) ) 2 ] E[(c,U(Q)) 2 ] 
E[(c,Q)(c,S(Q)-A(Q))J = + (33) 

+E[(c,Q + A(Q)-S(Q))(c,U(Q))] , (34) 
where U(Q) is the random vector of unused service (cf. in steady state. 



1G 



Proof. Recall the definition of W|(Q) — ||Q|||| 2 from (|16p and the subsequent definition of its one-step drift 
AW||(Q). We can expand this drift expression as follows: 

AW,|(Q) = [(c,Q[t + l]) 2 -(c,Q[i]) 2 ] X(Q[t] = Q) 
= (c,Q + A-S + U) 2 - (c,Q) 2 

= (c,Q + A-S) 2 + 2(c,Q + A-S)(c,U) + (c,U) 2 - (c,Q) 2 
= 2(c,Q)(c,A-S) + (c,A-S) 2 + 2(c,Q + A-S>(c,U> 
+ (c,U) 2 

Now, since ||Q|||| 2 < ||Q 2 || and E[||Q|| 2 ] < oo, we clearly have E[VF| 2 (Q)] < oo. Hence, in steady state, AWy 
must have a zero mean. Thus setting E[AW? (Q)] = gives the desired result. o □ 



The state-space collapse result is crucial in bounding the expectation in fl34[) . To provide some intuition, 
we first note the following useful property concerning the unused service. 

Lemma 9. For a queueing network evolving according to f7]) and for any given vector c y with ||cj| = 1, 
the following property always holds for any policy: 

(c,Q[t + l])(c,U[<]> = (-Q_L[i + l],U[t]) > (35) 

where Q_l[< + 1] = Q[i+ 1] - (c,Q[< + 1] ) c. 

Proof. We use the notation Q + = Q[t + 1] and omit [t] for the remaining parameters, Q[i], A[t], S[t], U[t], 
Accordingly, we define the projections and perpendiculars of Q + and U with respect to c as: 

Q+^(c,Q+)c, Ql = Q + - Q|J", U||=(c,U)c, Ux^U-Uy. 

The following argument uses the fact that Q^Ui = for all I, since either Ui[t] or Qi[t + 1] must be zero for 
each t. Hence, U is orthogonal to Q + . We can now prove the claim: 

(c,Q+)(c,U) = ||Q+||||U|||| ( = } (Q+ U,|) 

= (Q+-Q+ Uy) ( => (Q+.U-Ux) 

( = (Q+ + Q+ -U ± ) ( =' (-Q+ Ui), 

( = } (-Ql,u), 

where step: (a) corresponds to the equality case of the Cauchy-Schwartz inequality since Qjj~ >z and U|| >z 
are aligned by definition; (b) follows from the facts that _L Uy and Uy = U — Ujj (c) utilizes the facts 
that Q+ _L U and that Q + = Qj^ + Q^; (d) uses the fact that Qy" _L Ui; and (e) uses the facts that 
U = U_l + U|| and Q+ _L U|| . o □ 

Now, we present some intuition regarding the usefulness of the state-space collapse results (cf. Propo- 
sitions Q] and [2]). Using ([35} and noting that Q[t + 1] = Q[t] + A[t] - S[t] + V[t], we can re-write (JM)) 
as: 

m = E[(c,Q[i+l])(c,U(Q))] -E[(c,U(Q)) 2 ] 



< 



(a) 



E[(c,Q[t + l])(c,U(Q)>] 
E[(Q ± [< + 1],U(Q))] 



< ^E[||Q ± [i + l]|| 2 ]E[||U(Q)|| 2 ] 

i V /E[||Q ± P]E[||U(Q)|P], (36) 

where step (a) follows from Cauchy-Schwartz inequality and step (b) is true since the distributions of Q[t + 1] 
and Q[i] are the same in steady state. The last expression reveals the intuition behind why the state-space 
collapse results are useful: the expression vanishes as e J. since E [||QjJ| 2 ] is uniformly bounded, and unused 
service goes to zero as e J, 0. Next, we will build on this intuition to prove heavy-traffic optimality of both the 
JSQ Routing and MW Scheduling Policies. 
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5.1 Upper Bounds and Heavy- Traffic Optimality of JSQ Routing 

The following proposition presents a bound on the steady-state total queue-length under JSQ routing, and 
establishes the first-moment heavy-traffic optimality of JSQ as the network load approaches the capacity of 
the network. 

Proposition 3. Consider the routing problem under the exogenous arrival process {A^[t]} t with mean rate 
6 Int(7V) satisfying e = mx — As > 0, and with variance (a^) 2 . Then, under the JSQ Routing Policy, 
the limiting steady-state queue-length vector Q*' ^ satisfies 

L 



E 



.1=1 



< C(e) +B (e) 



(37) 



where we recall that C,^ = (<?^) 2 + Vy, as it is defined in Section \3.1\ and is o (- ) , i.e., ]imeB\ ' = 0. 



40 



Also, in the heavy traffic limit, where we consider a sequence of exogenous arrival processes {A^.\t]} t 



with e 4. so that A^ approaches /i£ and (o~^) 2 approaches a constant o\, we have 



lim sup el 

40 



(38) 



where £ = o\ + v\ . 

Hence, comparing the heavy-traffic lower-bound fTp for any feasible policy to the heavy-traffic upper-bound 
i38\ ) for JSQ Router establishes the first moment heavy-traffic optimality of JSQ Routing Policy. 

A 1 



Proof. Recalling the definition of c = -7- in the JSQ case, we first note a useful fact: 



E[(c,U(Q (e) ) 



Ms - As _ j_ 



(39) 



which follows from the fact that the mean drift of ( c, Q( £ ) ) must be zero in steady state. Next, we will 
temporarily omit the superscript for ease of exposition and study the terms in (|33j) and ()34j) under JSQ 
operation. 



E[(c,Q)(c,S(Q)-A(Q))] 



/ Ms - A; 



E[(c,Q)] = j-i 



.1=1 



(40) 



which follows from the independence of the total exogenous arrival process and individual service rate processes 
from the queue- length levels. 



E[(c,A(Q)-S(Q)) s 



1. 
V 



-E 



(As - 5s)' 



(a 2 + vl + e 2 ) 



(41) 



where we recall that o\ and v\ are respectively the variances of the exogenous arrival process {As[i]}t and 
the hypothetical total service process defined as Ss[t] = Ylf=i 



E[(c,U(Q)) 2 ] < (c,S , mox l)E[<c,U(Q) 



(42) 



where we use the bound Ui < S max for all I, and the identity (I39|) . 

Finally, to bound (j3"4")l , we take the same steps as in the argument leading to (f3T>|). and use the facts that 
c = \j\fL and Ui < S ma x for all I, to get: 



m < v e [ii^ii 2 ] e [ii u (Q)ii 2 ] 

< ^E[||QJ| 2 ]S mQX %/lE[(c,U(Q); 

< \/eN 2 VIS max , 
where, in the last step, we used (|39|) and N2 from Proposition [1] 



(43) 
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We reintroduce the superscript ^ to highlight the dependence on e and substitute P0|) - P5|) in 
to get, after minor algebraic manipulations, 



L 

l 



E 



((4 £) ) 2 + ^ + e 2 ) S max N 2 VLS max C (e) ^ ^ 



where C (e) = (o^) 2 + 4 + f2 and #1° - Ly N2 V ^ s — + which is o(l/e) as claimed. Then, (pf 

follows immediately by taking the limit of both sides. o □ 

5.2 Upper Bounds and Heavy- Traffic Optimality of MW Scheduling 

The following proposition yields upper bounds on the steady-state weighted total queue-length under MW 
Scheduling, and then establishes the first-moment heavy-traffic optimality of MWS as the network load 
approaches the boundary of the capacity region of the network. 

Proposition 4. Consider the scheduling problem under the exogenous arrival vector process {A^[t]} t with 
mean vector A^ 6 -* G Int(lZ) and with variance vector (cr( e ') 2 = ^(c^) 2 ^ . The parameter e — (e^ fc - ) )^ =1 >- 0, 
where gives the distance and the projection of X^ e ' to the k th hyperplane as defined in Ul\) . Then, under 

o 

A ( ' 

c«,Q (C) 



the MW Scheduling, the limiting steady-state queue-vector ^ satisfies, for each k G K° (£) 



E 



< ^+^ h \ ( 44 ) 



where we recall that £( e < fc ) = ( (c^-*) 2 , (er( e )) 2 ) as it is defined in Lemma\^ and is o^-^j, i.e., 

lim e^Bf' k) = 0. 

Also, in the heavy traffic limit, where we consider a sequence of exogenous arrival processes {A.^ e,k ^ [t]}t 
with their mean vector Ajf approaching the k th dominant face in ^^<C) along its normal c' fc ) , and variance 

vectors {c^) 2 approaching a constant vector cr 2 , we have 

limsupe^E \(c (k \Q {e) )] < (45) 



2 



2 a 2 



where C {k) = ( (c^) 

Hence, comparing the heavy-traffic lower-bound \15\) for any feasible policy to the heavy-traffic upper-bound 
J^5| ) for MW Scheduler establishes the first moment heavy-traffic optimality of MW Scheduling Policy. 

Proof. We temporarily omit the superscript ^ associated with the arrival and queue-length processes for 
ease of exposition, and consider any k E /C^. Before we embark upon the investigation of ([551) and (fM|) under 
MW Scheduler operation, we make several useful remarks. We first utilize the fact that the mean drift of 
( c' fc ) , Q ) must be zero under steady-state operation to get: 

E[(c( fe ),U(Q))] = (c( fe >, E[S(Q)])-(c( fe ),A) 

( =' (c( fe >,E[S(Q)]}-(6«-e«) 

C | e « (46) 

where (a) follows from and the fact that (c<», A (fc) ) = M fe ) since A (fc) € H [k) ; and (b) follows from the 
facts that E[S(Q)] must be in K and that ( c^, r ) < feW for all r G K by gj). 
Next, we define 



n W A 



((c( fe ),S(Q))=6W), (47) 



to be the fraction of time that the service rate vector is selected from face J 7 ^ by the MW Scheduler in 
steady-state. Also, we define 

7 (fe) = argmin{6 (fc) - (c (fe) ,r) : for all r G K\ F {k) }. (48) 
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Noting that ( c( fc ) , r ) = b^ k ' for all r £ J 7 ^ by definition and the structure of 1Z from ((U), we can see that 7^ 
measures the minimum loss in M fe ) for selecting a feasible rate r that is not on the face . It is important 
to note that 7^) is a strictly positive number with a constant value (independent of e) associated with the 
geometry of the capacity region 1Z. This constant helps us establish the following useful claim associated with 

7T«. 



Claim 1. For any £ (0,7 



A)) 



have 



c(fc) 



(49) 



where the upper-bound is 0{e^- k '), i.e., vanishes as e^ k ' 4- 0. 
Proof. (Claim [1} We start with the observation that 

E[(c«,S(Q)>] > (c« A) 



where the inequality follows from the stability of the queueing network (cf . Proposition [3j , and the equality 
follows from the utilization of (fT2|) . We can split the left-hand-side into two parts by using the definition of 
7r( fe ) to write 

7r W6W+E[<cW,S(Q))2'(<cW,S(Q))^6W)l > (& (fc) -e (fc) ), 

which, when re-arranged, leads to the following lower-bound on the expectation: 

E [(c( fe \S(Q))z((c (fc) ,S(Q)> ^6 (fc) )] > 6<*>(l-7r<*>)-e<*\ (50) 

Separately we can upper-bound the same expectation as 

E [(c (fe) ,S(Q))z((c (fe) ,S(Q)) =fe (fc) )] < (fe w - 7 ( ' £) )E[l((c (fe) ,S(Q)) ^ 6 (fc) ) 

= (6«-7 W )(l-^ fc) ), 

where the inequality follows from the definition of in (|48l) . and the equality from the definition of 7r^ fc ^ in 
((471) . Using this bound together with J50J yields (gSJ). o □ 



Claim Q] implies the following additional fact: 



E 



(&W _ < C «,S(Q) 



= (I-ttW)E 



< 



7 



(*■) 



(V fc )-(c«,S(Q))) I (<c« S(Q)}^6« 
((fo«) 2 + <cW,S mra l> 2 ) 



(51) 



<3(e (fc) ) 



where the inequality follows from (|4"9")l and the fact that Si < S max for all I. This result establishes in a certain 
probabilistic sense that ( c^ k \ S(Q) ) is close if is small. This result confirms the intuition that when 
is small, i.e., when A is close to the face F^ k \ the MW Scheduler must mostly select service rates on 

-p{k) 

so that the average service rate vector exceeds the given arrival rate vector componentwise to ensure stability. 

Our final remark before studying ([55]) and fM]) concerns the geometry of the scheduling capacity region 1Z. 
Since the number of possible rate vectors is finite, the number of faces in the rate region is finite. Therefore, 
for each face T^"' of the region 7Z, there exists an ang le e (0,7r/2] such that 



S(Q) 



for all Q £ {Q £ : ||Q , >\\ < ||Q|| cos((9 (fe) )} 



(52) 



where S(Q) is the service rate vector selected by the MW Scheduler for the given Q as in Definition [21 Note 
that 6>W identifies a cone around the line such that any Q in the cone leads to a rate allocation on the 
face F<® . 

We are now ready to study each term in ([33]) and (|34| with c := c^ k > to establish ((44)) . 
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E[(cW Q)(cW S(Q)-A(Q))] 



= E 



|| Q N ||1 (&<*> - <c<*>,A))-E [||Q||||(6 (fc) - (cW,S(Q))) 



® e (*) E 



> e (fe) E 



> e (fc) E 

(e) w 

> e W E 



> e (fc) E 



||Q||co S (0-- (fe) )(6W-(c( fe ),S(Q)» 



|Q|| cos(^ (fe) )I(^ (fc) > 0«)(&« - <c« S(Q) )) 



IQf II ™t(6^ m )l(9^ w > 8^){b^ ~ (c w ,S(Q) )) 



r(fc) 



Qr||X(^ >gW >(S>W)(6W-(cW,S(Q))) 



cot(6» (fe) ) 



|| Q W||(6W-(cW,S(Q)))J cot(0«) 
cot(6l( fc >) A /]E [lIQ^^I 2 ! E [(&(*) - (c( fe ),S(Q)» 2 

cot(0W 



,(fc) /u( fe ) 

iV 2 ((bW)* + (cW,S max l) 2 ), 



(53) 



where the step (a) follows from the definitions of in (ITT1) and of the angle 6'x.y between two vectors x and y 
given in ([2|); (b) is true from the definition of 0W; (c) is true since ||Q 



Q|| sin(0— — <fc) ); (d) is true since 
Wi w i 



cotangent function is a decreasing nonnegative- valued function in (0,7r/2]; (e) follows from Cauchy-Schwartz 
Inequality; and (f) follows from Proposition [2] and (|5"Tj) . We note that, in ([5"3")l . the first term is 0(e^ k ') while 
the second term is 0(V eW). 

Next, we bound the first right- hand- side term in ((33]): E [( c<», A(Q) - S(Q) ) 2 ] 



(a) 



«c<*>,A}-&< fc >) s 



■E 



(&«-<c« S(Q))) S 



+2 (( C W, A > - & w ) E [&W - ( c (fe) , S(Q) ) 



( i } E 

(c) 

< E 
(d) 

< E 



(e) 



((c( fc ',A) -&W) 1 



E 



(6W-(c( fc ),S(Q))) 5 



-2e^E 



& (fc) _ ( C «,S(Q) 



((c (fe) ,A-A) + (c (fe) ,A)-& (fc) ) ; 



■ E 



(&« -(cW,S(Q))) s 



(/) 



( , A - A > 2 + 2eW ( c« , E[A] - A > + (e^) 2 + E (6 (fe > - ( , S(Q) )) 2 
(fc) 

r 

((&«) 2 + (c«,S ma:c l) 2 ) 



[CI c\ ) / \ 

< < (cW)V 2 > + (6 (fc) ) 2 + — ((& (fe >) 2 + ( c( fc ),5 mQ:c l ) 2 ) 



y(fc) 



(54) 



where the step (a) follows simply from expanding the square after adding and subtracting b^ k \ and noting 
that the exogenous arrival rate vector A has mean A and is independent of the service rate vector S(Q); (b) 
follows from the definition of in (fTTj) ; (c) follows from adding and subtracting A in the first expression, 
and from noting that b^ — ( c^ k \r ) > for any r £ R and hence for S(Q); (d) follows, again, from the 
definition of e^; (e) follows from (|5"Tj) and uses the notation er 2 for the variance vector for the arrival process; 
and (f) uses the definition of the parameter £( e > fc ). Notice that all the terms except the first term vanishes in 
([54]) as | 0. 

Next, we bound the last term in (|33|) using (|46)) and the fact that Ui < S max for all I : 



E 



; c ( fe \U(Q)) 2 l < (c,S roax l)E[(c« U(Q))1 < e^(c^,S max l), 



(55) 



which is also vanishing as e^ k ' I 0. 
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Finally, we consider the term While the argument essentially follows that of (f56"| . we need to pay 

more attention to the zero entries of , which did not exist in the JSQ case. To that end, we define 
C++ = {I € {1, ■ ■ • ,L} : cj fe ) > 0} to denote the strictly positive entries of . Then, we focus on only these 
components by defining the following restricted vectors living in the \C++\ -dimensional real space: 

c<*> 4 (cj fc) V, ( *> , Q 4 (Q,). e ,w , U 4 (C70, e£ w • 



For convenience, we will denote Q[t + 1] = (Qi[t + l]),,- r (k) as Q + . In this reduced space, we further define 
the projection and perpendicular of Q + and U with respect to c( k > as: 

Q+ 4 (£( fc ),Q+)^ fc ), Q+4Q+-Q+, U|, 4^( fc ),U)cW, U ± 4U-U y . 

Now, since c^ fc ' >; satisfies ||c( fc )|| = 1, the statement of Lemma H] applies to c( k \ Q + , and U to yield 

(^ fc ),Q+)(£W,U) = (-Q+U). 

This result, together with the fact that (c^, Q + )(c ( - k \ U) = (c^, Q + }(c ( - k \ U), where we used the notation 
Q + = Q[t + 1], allows us to bound (|34p as follows (in the following we temporarily use Eq- to imply that Q 
is distributed as Q in the expectation): 

< Eq [(c( fc ),Q+)(c( fc ),U) _ 
= Eq[<£« Q+)(c« U) 
= (-Q+ U) 



(a) 
< 



(!>) 
< 



IQll 



lull 2 



J Q 



IQJI 2 



lull 2 



(56) 



where (a) follows from Cauchy- Schwartz inequality, and (b) is true since the distribution of Q + is the same 
as Q under steady-state. Next, we bound the expectations in (|56j) . 
The first expectation of (|56|) satisfies: 



E- 



Q 



IQj 



< E[||QJ| 2 ] < N?\ 



(57) 



where the first inequality follows from the fact that Qj^ and Qj^ are equal for all positions C++, while Qj^ 
potentially contains additional non-zero components, and hence cannot be smaller in magnitude. The second 
inequality follows from Proposition [5] that establishes the state-space collapse of MWS. 
The second expectation of (|5S)) satisfies: 



E- 



Q 



lull 2 



E- 



Q 



^ Smax g 



c« U(Q) ) 



„(*0 



(58) 



iec\ 



where c^] n 4 m in c$ > 0. Here, the last inequality is true since c[ k ^ < c^ in for all I e C++, and Ui < S v 
for all I. 

Substituting the bounds (f57j) and (|58|) back in (|56|) together with the fact (|46|) yields: 



< 



5ik)N {k)Sinax (5g) 



To complete, we reintroduce the ( e H e ' fe ) superscript to emphasize the dependence on e and substitute the 
derived bounds ([53]), JSJ), Ij55}, Q in d33J and flM|) to get 



E 



; c w,q (c) 
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where 



B 1 — cot 



( ; y e (fc) 7 (fc) 

(( 6 (*))2 



,(fe) c 



I) 2 ) , (c( fe ),5 mQ:c l) 



27W 



Ar( fe ) cf 
Jv 2 '-''max 

1 rain 



which is o(l/e^ fc ' ) ) as claimed. Then, immediately follows by taking the limit as e' fe '. 



□ 



6 Some Extensions of the Results on the Scheduling Problem 

In this section, we obtain bounds on the n th moment of the steady-state queue lengths for the scheduling 
problem, and we also discuss how to handle channel fading in the derivations of the bounds. Both of these 
extensions introduce some challenges, but can be essentially addressed by the methodology presented in the 
previous sections. The n th moment analysis can also be performed for JSQ routing, but is omitted here since 
it is quite similar to the scheduling case. 



6.1 n Moment Analysis 

We follow the steps outlined in Section [TTT1 to first develop lower bounds on the n th moment of steady-state 
queue- lengths, and utilize the state-space collapse result of Section [?] to find corresponding upper bounds. 
Then, we will establish, as before, the heavy-traffic optimality of these policies by showing that the appropriate 
dominant terms of the lower and upper bounds match as the arrival rate vector approaches one of the faces 
of the capacity region 1Z. 



1. Lower Bounds on the n th moment: We first extend the approach applied in Section [3] to the 
moment of the lower-bounding system in Figure El 



Lemma 10. For the system of Figured with a given service process {j3[t\\ t , consider the arrival process 
{a( e )[t]}t, parameterized by e > 0, with mean satisfying e — ft — and with variance denoted as o~ 2 a{e) . 
Let the queue-length process, denoted by {$( e ) evolve as in 0) with a[t] := a^[t}. 

Then, {$ (e) [t]}t is a positive Harris recurrent Markov Chain (iSOf ) for any e > 0, and converges in 
distribution to a random variable $ e with all bounded moments. 

Moreover, the n th moment o/$ can be lower-bounded as 



E 



> 



n > 1, 



(60) 



where B n e ^ vanishes with e i 0, i.e., limSi 6 - 1 = 

Therefore, in the heavy-traffic limit as the mean arrival rate approaches the mean service rate from 
below, i.e., as e \. 0, and assuming the variance cr 2 (c) converges to a constant a^, the lower bounds become 



liminf e n E 



> 



n > 1,, 



(61) 



where ( = <7 2 + v^. 
Proof. See Appendix [Cl 



□ 



Lemma flOl reveals an interesting fact that, in the heavy-traffic limit, the dominant terms of the n th moment 

of $^ only depends on £, which in turn depends only on the variances of the arrival and service processes. 
This is consistent with Brownian approximations, which utilize central limit theorem to approximate the 
system behavior using the first two moments of the arrival and service processes. 

We can now apply this generic result to the scheduling problem using the same construction as in Sec- 
tion! 
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Lemma 11. For the scheduling problem of Section \2.S\ with a given set of feasible schedules S, consider the 
exogenous arrival vector process {A { V[t]} t with mean vector A*- 6 -* G Int(lZ) and with variance vector denoted as 

(er( e )) 2 = ^(c^) 2 ^ • Here, the superscript vector is given by e — (e^)jLi >- 0, where and gives the 

distance and the projection of X^ to the k th hyperplane as defined in h 11)) and A 12)) . respectively. Accordingly, 
let the queue-length process under MW Scheduling with this arrival process be denoted as {Q( e '[i]} t , evolving 

as in (Qp. Moreover, let Q ^ denote a random random vector with the same distribution as the steady-state 
distribution of {Q^ [t]}t (by Lemma\B^. 

Then, for each k G {1, • • • , A"}, and with C (e ' fe) = ( (c< fe )) 2 , (er( e )) 2 ), 

r i //(e,t)\ n 
( e W) n E (c^Q 1 } )" > n! -B^ k \ n > 1, (62) 



where B^' k ^ vanishes as e^ k ) \. 0. 

Further, for each k G /C»(e), consider the heavy-traffic limit e( k ' I 0; and suppose that the variance 

vector (<r( e )) 2 approaches a constant vector a 2 . Then, defining = ( (c^) 2 ,cr 2 ), we have 

liminf(e (fc) )"E [(c< fc \Q (e) ) n j > n\ ( , n > 1. (63) 
c( fc )^o L J \ 2 J 

2. State-Space Collapse: The state space collapse result for MW Scheduling provided in Proposition [2] 
applies directly. Next, we utilize this result to develop upper bounds on the n th moment of the steady-state 
queue- lengths. 

3. Upper Bounds and n th Moment Heavy- TrafHc-Optimality of MW Scheduling: The main idea 
behind the analysis is to utilize the state-space collapse result, which implies that the total unused service 
under the MW Scheduler is small unless all queue-lengths are small. Thus, as the system gets heavily loaded, 
the total unused service vanishes, making the system act similarly to the lower bounding system investigated 
above. The following result builds on this to establish the n th moment heavy-traffic optimality of MW 
Scheduler. 

Proposition 5. Consider the scheduling problem under the exogenous arrival vector process {A^[t]} t with 

mean vector A^ 6 -* G Int(lZ) and with variance vector (er( e ') 2 = ^(c^) 2 ^ • The parameter e — (e^)j^_j >~ 0, 

where gives the distance and the projection of X^ to the k th hyperplane as defined in Ul\). Then, under 
the MW Scheduling, the limiting steady-state queue-vector Q 6 satisfies, for each k G ^-^(6) j 

( e W)«E[(c« Q (6) )l < nl(^) +B { n e ' k \ n>l, (64) 



where we recall that £( e '*-') = ( (c^) 2 , (<x( e )) 2 ) as it is defined in Lemma® and B^' k ^ is vanishing as e^ k ) 4- 0. 
Also, in the heavy traffic limit, where we consider a sequence of exogenous arrival processes {A^ € ' k '[t]}t 



with their mean vector X^ approaching the k th dominant face in K.° (£) along its normal , and variance 



vectors (o^ ) 2 approaching a constant vector cr 2 , we have 

,(*) n (e) 



limsu P (e( fc )) n E ( C W,Q V ; ) < n! ( ) , n > 1, (65) 



where C [k) = ( (cW) 2 ,<x 2 ). 

Hence, comparing the heavy-traffic lower-bound i6St)) for any feasible policy to the heavy-traffic upper-bound 
i65)) for MW Scheduler establishes the n t/l -moment heavy-traffic optimality of MW Scheduling Policy. 

Proof. See Appendix [DJ o □ 

We note that the n th moment argument concerns optimality of the norm of the projection HQ^H onto 
the vector c( fc ) . It is interesting to relate this norm to the norm of the queue-length vector || Q|| when n = 2 to 
show the 2 nd moment optimality of MW Scheduling in minimizing lim £ (fc)^ (e( fc )) 2 E[||Qj| 2 ]. This consequence 
is provided in the following corollary. 
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Corollary 1. Under the same conditions as in Proposition^ the MW Scheduler achieves, in the heavy-traffic 
limit, 

limsup( £ W) 2 E[||Q|| 2 ] < (66) 

where C' fc ) — ((c^) 2 , cr 2 ) . Furthermore, since the right-hand- side is the smallest heavy-traffic limit achievable 
by any policy, this establishes 2 nd -moment heavy-traffic- optimality of MW Scheduling. 

Proof. The proof simply follows from noting that 



|Q! fc) ll 2 <l|Qll 2 = l|Qfi fc) ll 2 + llQi fe) ll 2 , 



from Pythagorean Theorem, and from combining the state-space collapse result E[||Q^|| 2 ] < where 

N% is independent of with the tight heavy-traffic lower and upper bounds (|6ip and (|65[) on E[||Qn || 2 ]. 
o □ 



6.2 Channel Fading 

We consider the same setup as in Section l2~3l depicted in FigureEJ receiving nonnegative- integer-valued vector 
of arrivals {A[i]} t >o with Ai[t] < A max ,yi,t, distributed independently over links and also identically over 
time. However, instead of a fixed set of feasible rate vectors S, we allow the feasible set to evolve randomly 
in time over a finite state space. In particular, we let }t>o be an i.i.d. sequence of random variables 
capturing the global state of the channel states of all links in the network. We assume that J[t] £ J for 
some set J with finite cardinality, and let tjjj — F(J[t] = j). Then, each global state j € J yields a set of 
feasible rate vectors that can be provided under that state. We assume that S^> has finite cardinality for 
each j with Si < S max , for all l and each S = (£/); € S^K For simplicity, we also assume that each feasible 
rate vector in S^' is composed of non- negative integers, although the arguments hold for any discrete set of 
choices. 



Capacity Region Under Channel Fading: With channel fading, the maximum achievable rate region 
becomes 

K = Convex Hull (S u) ) 

iej 

= Convex Hull < ^ ipj s (j) : s (j) g <S (j) , for each j g J 

which is henceforth called the Fading Capacity Region. Notice that the set Convex Hull (<S^) for each j g J 
simply yields a convex polyhedral set in as in Thus, their finite weighted-sum also yields a polyhedral 
set that can be equivalently described, with a convenient abuse of notation, as: 

K = {r>0: (c< fe \r) < b^ , k = 1, • • • , K}, (67) 

where K denotes the finite (and minimal) number of hyperplanes that fully describe the polyhedron. We 
refer the reader to Definition for the notions of the hyperplane H.^, the pair (c^jb^), and the face J 7 ^', 
which identically apply to 1Z. 

We note that the explicit characterization of 1Z is not necessary for the rest of our argument. Nevertheless, 
we present an example with ON/OFF fading to clarify the nature of 1Z. 

Example 1 (Capacity of Downlink Communication over On/Off Fading Channels). Consider the scenario 
of Figure [7] where a wireless base station serves L independent flows over fading ON/ OFF channels. While 
the channels in OFF state cannot transfer any information, channels in ON state can successfully transfer a 
packet in a slot. Yet, due to interference, only one of the ON channels can be used in each slot. We assume 
that the channel state vector {Ci\t\)i is known at the beginning of each slot t, and vary independently over the 
links. For each I, {Ci[t]} t >o are i.i.d. over time with some pi = P(Ci[t] = 1) = 1 — P(Ci[t] = 0) G [0, 1]. The 
corresponding capacity region 1Z DL for this fading downlink scenario is explicitly characterized in \39^ as: 



n 



DL 



J r h : 5>/ < 1 - llil-Pi), for all Q C {1,- •• , L] 
I leg leg 
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Figure 7: Downlink Communication Setup over 
ON/OFF Fading Channels 



Figure 8: The fading capacity region 1Z for L = 
2 with pi = 1/2 and p2 — 1/3. Note that for this 
case: (cW)| =1 = ((0, 1)', (1,0)'), and 

(& (fe) )Li-(lAV2/3,l/2). 



j4ti example instance of this region is depicted in Figure^ for L = 2 with pi = 1/2 and p 2 = 1/3- fig ure 
also demonstrates how the statistics of the channel influence the characterization of the hyperplanes associated 
withK DL . o 

Maximum Weight (MW) Scheduler Under Fading: In each slot, the purpose of the scheduler is to 
select a feasible rate vector from the feasible set to achieve stability of the queueing network in the long run. 
A well-known generalization (introduced in [35]) of the earlier MW Scheduler (cf. Definition [2]) to this fading 
case is the following: given the queue-length vector Q[£] and the global channel state J[t] at the beginning of 
slot t, the service rate vector is selected as 

S[t] = S(Q[t], J[t\) = RAND \ argmax(Q[i],S) > . (68) 

Note that the rate vectors in 1Z are typically not instantaneously realizable, but only in the mean sense. 
In particular, mean service rate vector over the channel variations provided by the above MW Scheduler 
conditioned over a queue-length vector Q satisfies: 

R(Q) = E[S[t] | Q[f] = Q] = RAND jargmax ( Q, R ) 1 . (69) 



We next comment on the application of the three steps of our methodology outlined in Section 11.11 to the 
steady-state performance of this MW Scheduler in the fading scenario. 

1. Moment Bounds and Lower Bounds under Fading: The statement and proof of Lemma [3] apply 
once we replace the non-fading capacity region with the above fading capacity region. However, we note that 
the specific values of c( fe ) and b^ k ' are different in the non-fading and fading capacity regions, as the latter 
also incorporates the channel fading distribution ip = {if)j)j- 

Next, we construct the lower-bounding system associated with the k th face of 1Z, for any given k € 
{1, ■ • • , K}, as in the non- fading case, except that the service process is no longer at a constant rate but must 
incorporate the channel fading distribution if). Recalling that the pair (c^ k \b^) describes the associated 
hyperplane, we first define 

= max ( c (fe) , s ) , for each j € J, 

which yields the maximum -weighted service rate available in channel state j. 

We are now ready to describe the governing arrival and service statistics of the lower bounding system 
(cf. Figure[5]): the arrival process {a^ [t]}t>o of the lower bounding system associated with hyperplane R^ 
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is set to a( fc )[i] = ( c^ k \A[t] ), while the service process [t]}t>o is distributed as: 

P (/?( fc ) [t] = b( j < k A = ipj, for each j G J. 

We note that {b^' k '}jeJ may be identical for different j, in which case their probabilities are aggregated. It 
is also true that — E[/jW[t]] by construction of 1Z. 

We note that the queue-length process {<t>^ & ^ [t] } t >o driven by the above arrival {a^ fe ^[t]}t>o and service 
{(3^[t]}t>o processes as in JSJ) is stochastically smaller than {(c^ k \Q[t] )}*>o, where {Q[i]}t>o is the queue- 
length vector process under any feasible scheduling strategy. This follows from a coupling argument that 
utilizes the facts that: the service process of both the lower bounding and the actual queueing systems are 
governed by the same fading distribution ip] and that b^' k \ by definition, is the largest c( fc ) -weighted service 
that can be provided to the queueing system under channel state j. Hence, any lower bound on [f]}t is 
also a lower bound on {( c^ k \ Q[t] )} t . 



The setup and definitions of Section 14.21 apply directly to the fading case given that 1Z is the fading 
capacity region defined above rather than the the non-fading capacity region of Definition [3] Then, we have 
the following equivalent of Lemma [6] 

Lemma 12. For the above scheduling problem with a given set of channel fading distribution xp = (ipj)j and 
their associated set of feasible schedules {S^}j, consider the exogenous arrival vector process {A^[t]} t with 

mean vector X^ G Int(lZ) and with variance vector denoted as (cr^) 2 = y(crj e ' ') 2 ^ . Here, the superscript 

vector is given by e = (e^)j(Li >~ Oj where and gives the distance and the projection of X^ to the k th 
hyperplane of J6'7| ) as defined in Ml)) and MS\) . respectively. Accordingly, let the queue-length process under 
MW Scheduling (see H68\) ) with this arrival process be denoted as {Q( e '[i]} t , evolving as in {IJ). Moreover, let 

Q,^ denote a random random vector with the same distribution as the steady-state distribution of {Q^ [t]}t- 
Then, for each k G {!,••• , K}, we have 



E 



:«,Q (e) 



where ( {e ' k) = E \(( c«,A( £ ) > - /3«) 2 j = ( (c«) 2 , {(T^) 2 ) + Var{fi^) + (e^) 2 , and B (k) 4 



2 

(*0 



Further, for each k G /C, (6) , consider the heavy-traffic limit ' \. 0; and suppose that the variance 
vector (er^ 6 -*) 2 approaches a constant vector a 2 . Then, defining (} k ^ = ( (c^) 2 , cr 2 ) + Var(j3^ k '), we have 



lim e (fe) E 



; c w,Q (e) 



> v 



Comparing these lower bounds with their non- fading counterparts (|14p and (1151) , we note that they include 
Var((3^ ) which captures the impact of the fading distribution ip on the steady-state mean queue-length levels. 

2. State-Space Collapse of MW Scheduling under Fading: The state-space collapse argument under 
fading follows the same line of argument as in Section [4] First, the statement and proof of Lemma [7] applies 
without modification since the maximum service rate is uniformly bounded by S max in every channel state. 
Next, we follow the development of Section l4~2l to consider the performance of a sequence of systems associated 
with a sequence of arrival processes {A^ e ^[t]} t > parameterized by e = (e^ k ')k >- as defined in (fTT|) . 

Then, the corresponding queue-length vector process under MW Scheduling {Q*- e '[i]}t converge in distri- 
bution to c/ 6 '. Also, the definitions of Q[j e ' fe \ Q^ e ' fe ' ) , and ^w€) remain the same as in Section l4~2l with 1Z 



representing the fading capacity region ([67]) . Then, the following state-space collapse result follows. 
Proposition 6. Assume A*- 6 -* G Int(JZ) in the scheduling problem with the corresponding set of {e^}k as 

"A (i 



defined in ill]) . Then, under the MW Scheduling Policy, for each k G JC° (C) , there exist finite constants 



{Nr k ^} r =i,2, -- such that E ||Qj_ ' '|| r < Nr" 1 , for all e >- 0, each k G /C° , e , , and each r = 1, 2,-- ■ . 

Outline. The proof of Proposition [5] directly applies to this statement with 1Z defined as in (I67[) . and by 
replacing E[S | Q] with R(Q) as defined in (p^)) . Most importantly, with these substitutions, the key property 
(|30|) continues to hold for the MW Scheduler under fading, which allows the rest of the argument to apply 
without modification. o □ 



i7T( e . fc ) 



r (A0 
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3. Upper Bounds and Heavy- Trafflc-Optimality of MW Scheduling under Fading: Similarly 
to the first two steps, the upper bound arguments of Section [5] extend to the fading scenario with minor 
modifications. First, Lemma [5] applies once we replace S(Q) with S(Q, J) (see ([55)) ) to capture the channel 
randomness, and let the expectation be over the channel fading distribution xp as well. Then, the statement of 
Proposition 2] applies almost without modification to the fading case, which is repeated here for convenience. 

Proposition 7. Consider the scheduling problem under fading with capacity region ffi7| ) serving the exoge- 
nous arrival vector process {A^[t]} t with mean vector X^ £ Int(7Z) and with variance vector (er^ 6 )) 2 = 

^((r| e ') 2 ^ . The parameter e = {e^)^ = i >- 0, where e( fe ) gives the distance and the projection of X^ to the 

k hyperplane oflZ as defined in ill)) . Then, under MW Scheduling, the limiting steady-state queue-vector 

Q "* satisfies, for each k G JC° (C) , 

A 



E 



^,Q (£) >] < ^ + Bf k \ (72) 

(e,fc) 



where we recall that C (e ' fe) = ( (c^) 2 , (er( e )) 2 ) + Var(/3^) + (e^) 2 as it is defined in Lemma\M and B\ 



^),i.e, Jim^M e ' fc) =0. 



Also, in the heavy traffic limit, where we consider a sequence of exogenous arrival processes {A\ ' )\t]}t 
with their mean vector X^ approaching the k th dominant face in K° (e) along its normal c' fe ' , and variance 

vectors (c^) 2 approaching a constant vector cr 2 , we have 



lim e (fe) E 



; c «,Q (e) 



< V' (73) 



where C (fc) = ( (c^) 2 , (<x( e )) 2 ) + Var{[3^). 

Hence, comparing the heavy-traffic lower-bound |7_?| ) for any feasible policy to the heavy-traffic upper-bound 
f73p for MW Scheduler establishes the first moment heavy-traffic optimality of MW Scheduling Policy under 
fading. 

Outline. We point to a few modifications in the proof of Proposition |4] that yields the proof of this statement. 
The need for this modification arises from the fact that the MW scheduler does not directly select its allocation 
from the fading capacity region TZ, but from the instantaneous feasible set of schedules {S J ^}t available at 
the time. This subtlety can be handled partly by working with conditional expectation R(Q) defined in 
instead of S(Q), and partly by introducing conditional probabilities in the analysis. 

Omitting the details, we point to the similarities and differences from the non-fading case: the derivation 
of (j46]l follows with S(Q) replaced by R(Q); the definition of tt^ in (|47|) is modified to per channel state j 
as 

n U,k) A p h c (*) > s(Q, J) > = b^^ \J = j 



for each j, k; the definition of y( k ' in (|48l) is unmodified; the statement of Claim Q] is true for the above 

conditional probability ^j k \ such that (1 — 71"]^) = 0(e^) for each channel state j. This establishes that 
under heavy-traffic conditions, the MW Scheduler will operate on the dominant face with high probability in 
every channel state. The rest of the argument follows the non-fading case since the previous fact allows us to 
approach the fading case as a time-average of non-fading cases, each operating on rate vectors contributing 
to the dominant face J 7 ^. o □ 



7 Conclusions 

The main contribution of the paper is to show that drift conditions in steady-state can be used to obtain 
bounds on the moments of queue lengths that are tight in heavy traffic. The key new idea here is to derive 
an appropriate notion of state-space collapse in steady-state which sharpen the bounds obtained using drift 
conditions. The results presented in this paper apply to the case where the state collapses to a single dimen- 
sion. An interesting topic for further research is to understand whether the ideas presented here apply more 
generally. 
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A Proof of Lemma [2] 

The proof follows from the application of Lemma Q] to the Markov Chain X[t] := Q[t] using the Lyapunov 
function Z(X) :— V(Q) = ||Q||. First, we check that both conditions (CI) and (C2) are satisfied. We start 
with (CI): 

E[AV(Q) | Q[*] = Q] = E[||Q[t + l]||-||Q[t]||]|Q[t] = Q] 

= e [VIIQ[* + 1]|| 2 - VIIQMIP I QM = Q 

" ^£[||Q[t+l]|| 2 -||Q[i]|| 2 |QM = Q], (74) 

where the inequality follows from the fact that f(x) — \fx is concave for x > so that f(y) — f(x) < 
(y - x)f'(x) = with y := \\Q[t + 1]|| 2 and x := ||Q[£]|| 2 . Next, we study the difference in (74]), which is 

simply the mean drift of the quadratic Lyapunov function W(Q) = ||Q|| 2 - We shall omit the time reference 
[t] after the first step for brevity. 

E[AW(Q)|Q] = E[||Q[i + l]|| 2 -||Q|| 2 |Q] 

= E[||Q + A-S + U|| 2 -||Q|| 2 |Q] 

= E[||Q + A-S|| 2 + 2(Q + A-S,U) + ||U|| 2 - ||Q|| 2 |Q] 

< E[||Q + A-S|| 2 -||Q|| 2 |Q] 

= E[2(Q,A-S) + ||A-S|| 2 |Q] 

< 2E[(Q,A-S) | Q] +JjTi, (75) 

where the inequality (a) follows from the fact that Ui(Qi + Ai — Si) = —U? < 0, for each I, and K\ = 
Lm&x(A max , Smax) 2 is finite since both the arrival and service processes are bounded. 

Next, we bound the first term in (f75|) by first defining e = /is — As, and then defining a hypothetical 
arrival rate vector A = (A/)/ with respect to the given service rate vector fj, and e such that A; = (Xi — 4, for 
each I. Note that A; = Ms — e = As- Then, we can massage the first term in (|75j) as 

E[(Q,A-S)|Q] = (Q,E[A|Q]-A)-<Q,/i-A> 
( = } (Q,E[A|Q]-A)-1<Q,1> 

= E[A S | Q] Q m i n - ( Q, A ) - -^-HQHi \ Qrnvn = min Q m > 

L I l<m<L 

L 

= As Qmin — MQl — Y HQll 



1=1 

L 



J2 X l(Ql- OtninJ-fllQll 



1=1 



< - I HQ|| 

where step (a) follows from the definition of A; (b) follows from the definitions of the JSQ policy (see Defini- 
tion!]]) an d the l\ norm ||Q||i = Yli=i \Ql\> ( c ) ^ s trivially true since Q m in < Qi for all I. Using this bound in 
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frS]) and back in §7%$) yields 



E[AV(Q) | Q[t] = Q] < -I ' K 



L 2||Q||' 

which verifies Condition (CI). Moving on to Condition (C2), we have 

|AV(Q)| = |||Q[< + f]||-||QM|||I(QM = Q) 
= ||Q[t + l]-Q[t]||X(Q[i] = Q) 

< ||Q[t + 1] - Q[t]||iX(Q[t] = Q) 

< L max |Q, [t + 1] - Qi [t] |X(Q[t] = Q) 

(6) 

< 2L max(A max , S max ) (76) 

where (a) follows from the fact that ||x|ji > ||x|| for any x € R , and (b) is true since Ai[t] and Si[t] are 
bounded by A max and S max , respectively, for all / = 1, • • • , L and all i > 0. This verifies Condition (C2) and 
completes the proof. 

B Proof of Lemma [7] 

We first prove l[2"0]): 
AFl(Q) = [||Qx[i + l]||-||Qx[i]||]I(QM = Q) 



< 



v/||Q ± [i + i]|| 2 - v/IIQ^WII 2 ] z(QM = Q) 
1 



2||QxM|| 
f 

2||Qx[t]|| 



[||Qx[i + l]|| 2 -||QxWH 2 ] I(Q[t] = Q) 
[ (||Q[i + 1]|| 2 - ||Q[t]|| 2 ) X(Q[t] = Q) - (||Q|| [t + 1]|| 2 - ||Q|| [t]fl(Q[t] = Q)) ] , 

S v ' V v ' 

=AW(Q) =AWy(Q) 



where the inequality follows from the fact that f(x) = y/x is concave for x > so that /(y) — /(x) < 
(y — x)f'(x) = %^7=r with y := 1 1 Q j_ [t + 1]|| 2 and x := \\Qj_[t}\\ 2 . Also, the last step follows from Pythagoras 
Theorem © with x := Q||[-] and y := Qx[-]- 
Next, we prove (|2T|) : 



|AVx(Q)| = |||Qx[t + l]||-||QxM|||2:(QM = Q) 

(a) 

< ||Qx[i + l]-Qx[i]||X(Q[t] = Q) 



\Q[t + 1} - Q[t] - Q,i [t] + Qi, [t + 1]|| X(Q[t] = Q) 



< (||Q[i + l]-Q[<]|| + ||Q||[i + f]-Q||[i]||) X(Q[t] = Q) 

(d) 

< 2||Q[i + l] - Q[i]||X(Q[i] = Q) 

(e) _ 

< 2VL rn^c |Qi[t + l] -0/[i]|I(Q[t] = Q) 

(/) r 

< 2v L max(A maa; , S max ) 

where: (a) follows from the fact that | ||x|| — ||y|| | < ||x— y| for each x, y <E K L ; (b) follows from the definition of 
Q = Qj^ + Q|| ; (c) follows from triangle inequality; (d) follows from the non-expansive nature of the projection 
onto a convex set once we note that Q|| [•] is the projection of Q[-] onto the line along c, which implies that 
||Q|| [t + 1] — Q|| [t]\\ < ||Q[i + 1] — Q[t]||; (e) trivially follows from the definition of || • ||; and (f) is true since 
Ai[t] and Si[t] are respectively assumed to be bounded by A max and S max for all I = 1, • • • , L and all t > 0. 
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C Proof of Lemma ITU 



We have already argued the weak convergence and boundedness of all moments of the limiting random variable 
$^ for each e > in the proof of Lemma 2] Recalling the evolution ©, we study the mean drift of the 
Lyapunov function W n {<&) = ||$||". In the following, we temporarily omit the time reference [t] and the 
superscript ( e ) for ease of exposition: 



i[AW„($[i])| = = E[($ + a-/? + x)"-$ n |$] 



= E 



® E 



($ + a - f3) n - $ n + 

i=0 
n-l 

($ + a - p) n - $" + X n E 



($ + a -/3) l x "- l |$ 



(-!)<!* 



E [(* + a - 0) n - - (~ x ) n I *] 
rvi-i 



= E 



E 



E 

n-2 

E 

i=0 



{-xT I * 



*'(a - p) n - % + n$"" x (a - (-%)" | $ 



where (a) uses the fact that + a — + a — (3 < 0) by definition of Taking expectations of both 

sides with respect to the steady-state distribution, i.e., setting $[£] = <I>, and noting that E[AW / rl ($)] = 
yields: 







E 



E 



Re-arranging terms, noting the independence of the arrival and service processes from each other and and 
recalling that E[/3 — a] = e by construction allows us to write: 



n-2 , s 

1 J2( " E [(a -/?)"-] -E[(- X )»] 

We separate the final term in the summation and multiply both sides with e n ~ 2 /n to get: 
J JEM>" 'I = — — ("' -W "]E[(a - jS) 2 , 

e (n-2) E ^_ x y 



neE\<P 



"- 1 E[¥ n " 1 ] = ( " ~ 1 ) e ("- 2 )E[$"~ 2l1g ' ,J 

n — 3 



Next, we investigate terms in ((75)) to show that they vanish with e 4 0. To that end, we first note that 



5 ("-2) E [(_ x )r 



< 



-(n-2) C"- 

If) / 



EM = 



c(n-l) an-1 



(77) 
(78) 

(79) 



where the final equality follows from the fact that E[x] = e under steady-state operation. Clearly, the final 
expression vanishes with e 1 for any n > 2. 

Next, we turn the summation in (|78p to argue inductively that lime™~ 2 E[$ 4 ] = for all n > 3 and for 

each i = 0, 1, • • • , n — 3. This result, when proven, implies that the whole sum is vanishing with e. The first 
step of the induction holds trivially since eE[$ ] = e. Suppose the claim is true for some n > 3 and all 
i = 0, 1, • • • , n — 3. We would like to confirm it for (n + 1) and i = 0, 1, • • • , n — 2 as well. This is straight- 
forward for all i = 0, 1, • • ■ , n — 1. The case when i — n — 2 requires us to investigate e" _1 E[<I>™ ] using the 
expansion ([77)1 -(1781: 



X E[$ 



.—n-2. 



E 



5[f 



E [(a - 



2 E[(-x)' 
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The right-hand-side of this expression vanishes with e since e^^E^'] vanishes for each i = 0, 1, • • • , n — 3 by 
the induction hypothesis, and e" _2 E[(— x) n ] vanishes according to (l79l) . We note that the moments of (a — (3) 
are bounded since the arrival and service processes are both bounded. This completes the induction proof, 
and hence establish that lim e n_2 E[$ i ] = for all n > 3 and for each i = Q, 1, • • • , n — 3. 



f.-|-0 



Returning to ([77 j) -([75 ]l and revoking the ^ notation to highlight the dependence on e, we have thus proven 
that there exist {Cn }n>i that vanish with e such that 

e"E[($ (e) ) n ] > |e("- 1 )E[(¥ (e) )" _1 ]IE[(a (e) -pf]~C^ 



e (»-i)E[($ (e) ) n - 1 ] - cfi 



(80) 



where — (c^) 2 + He + e2 i an d — J2k=i ( n -k)\ ^n-fe a ^ so vanishes with e, which proves (j60 

The heavy-traffic result (ICTj) then follows immediately by taking the limit of both sides as e | 0. 



D Proof of Proposition [5] 

We temporarily omit the superscript ( e ) associated with the arrival and queue-length processes for ease of 
exposition, and consider any k G /C^. We, then, study mean drift of the Lyapunov function Wn k \Q) — 
( c( k \ Q )™ associate with the n th moment. 



E[AWW(QM)|Q[t] = Q] = 



c<*> ,Q + A- S + U) n -( c (fc) ,Q )™ | Q 
[( C ( fe ),Q + A-S)"-(cW,Q)"|Q 



+E 

e[«. 

i=0 

E[((c 

71-1 

+E 

i=0 

71-1 

+E 

4=0 
71-2 

+E 

i=0 
n—1 

+E 

i=0 
71-1 

+E' 

i=0 



(fc) 



c( fc \Q + A-S) l (cW,U)"- 1 | Q 
,Q + A) -6 (fe) +6 (fe) - < C W,S)) n - < C W,Q}" | Q 



E 



;c (fc \Q + A-S) j; (cW,U) n ^ ; | Q 



»,Q + A>-6 (fe) )"-(c (fc ),Q)"|Q 



E 



(( c (fe) ,Q + A > - &(*))<(&(*) - ( C W, S ))™ | Q 



C W,Q + A-S) l (c( fc \U)"- 1 | Q 
Q)"- 1 E[( C W,A) -fe( fe ) | Q] 

cW,Q) l ((c«,A) -&(*>)"-< | Q 



(( c (fc) ,Q + A ) - &W) l (& (fc) - ( c (fc) , S )) n - 1 | Q 
(c (fe \Q + A-S) ,; (c (fe \U)"- ,; | Q 



(81) 



Note in (jgTj) that E[( c (fc ) , A ) — 6 (fe ) | q] — _ e (*0 by construction. Thus, taking the expectation of both sides 
over the steady-state distribution, i.e. setting Q = Q, noting E[AVKn fe ^(Q)] = 0, and multiplying both sides 
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with (eW)™ 2 yields: 

n-2 

n(e (fc) )"- 1 E[(c (fc) ,Q)™- 1 ] = ^ 



i=0 
n-1 



( £ (fc))n-2 E L C (*) 5 Q )*(( c (*) > A ) _ b (k)y 



(82) 



E 

i=0 
n-1 

E 



0. 


) (e (fe) )™ 


~ 2 E 




) (e (fc) )™ 


~ 2 E 



C W,Q + A-S(Q)) 4 ( C W,U(Q) 



&(*))*(&(*) - ( c W,S(Q))) n -(8l3) 



(84) 



We note the resemblance of this expression to (I77[) - (I78I) of the lower-bounding system with $ replaced with 
( c^ k \ Q ). Accordingly, the claimed upper bound (|M1) follows from induction as in the argument (|8H|) once 
we establish that (l83l) and (|84|) both vanish as e^ fc ^ 4- 0. Thus, we next study the behavior of each of these 

expressions with e'" for all n > 2. 

We first study the expectation in ({83} to show that it is of order Ve" and hence (|83|) must vanish as 
e( fc ) I for any n > 2. For any i = 0, 1, • • ■ , n — 1, we have 

E [(( cW, Q + A ) - &«)*(&( fc ) - ( c( fc ), S(Q) ))»-*] 

"(cW,Q) J «c(* ) ,A)-6W) J - < (6 w - (c (k \S(Q))) n ~ 



E 

3=0 

i 

E 

3=0 






j E 













] E 







||Q[ fe )||j(5(fe) _( C « ) S(Q)}H E \((c (k \A) -b (k *>y 



(85) 



where in the last step we used the independence of arrival processes, and used the definition of Qy . Following 
the same argument as in the derivation of (1531) to bound the first expectation as: 



E 



||Q[ | * ) ||J(&(*)_( c W,S(Q))) n - < < (cot(0<*>))'E \\Q ( * } \\ j (bW - ( C W,S(Q))) 



< (cot(6» (fe) )) J \/E 



E[(M fe )-(cW,S(Q))) 2 ("- 1 )] 



We know from Proposition [2] that E 
i G {0, • • • , n — 1}, we can show that 



lQi fe) ll 2j 



E 



(&(*) _ ( c ( fc ),S(Q))) 2 ( n - 1 ) 



< 



7 



< AT 2 ( *° for some finite independent of e^ k \ Also, since 

((jW)2(«-) +(c W i 5 mnil )2(n- I )j = ( e W) 



exactly as argued in (jSTj) . Thus, these two bounds together with the fact that Ai < A max for all / establishes 
that ||S5J| = O (VeO^j , which, in turn, proves that (P|) = O f(e<») n -§) , i.e., vanishes as e<» | for all 
n > 2. 

Next, we study the expectations in ([54")) to show that they also vanish as e' fc ) J, for all n> 2. We recall the 
c, Q, U, Q + , and Eq[-] notation introduced for the derivations ([55 ]) - ([5^)) and follow the same line of reasoning. 
In particular, we note that, for each i = 0, 1, • • • , n — 1, the expectation in (|84[) can be bounded as 



(c (fc) ,Q + A - S) 4 (c (fc) ,U)™~ 4 



< % 



E 



Q 



E 



Q 



(c( fe \Q+) l (c( fe \U)"- 1 
( cW,Q + U}"^ 

iiQliiiiuiiir- 



(86) 



To bound (|86p . we consider two cases separately: i < n/2, and i > n/2. When i < n/2, the argument closely 
follows ([55| - ([5^)l to show that (|56")) = O fV eP^j . When i > n/2, we multiply and divide (assuming the 
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non-trivial case of ||U|||| ^ 0) by ||Um 



to get 



E: 



Q 



Q 



+11*1 



Ui, 



= E 



(a) 



Q 



Q 



XJn 



l|U|||| 21 - 

:Qt,u; 



E 



Q 



l|U|||| 

Qi, 



2i-n 



u 



IUn 



12-^ 



(b) 
< 



E- 



Q 



QI 



+ ||2z 



-Q 



||U||2(n-i) 



where (a) follows from Lemma |H] similarly as in the derivation of (1561) ; and (b) follows from Cauchy-Schwartz 
inequality after minor modifications. It is then easy to show, as in (|57| and ([58]), that Eq ||Qi|| 2i 



< TV, 



(fc) 



with defined in Proposition [2] and E- 



Q 



U||2(n-i) 



o 
o 



/e( fc 



— O(e^) since i < n— 1. Hence, we have shown that 
in this case as well. Substituting this result back in (|M|) proves the claimed result that 

ij , i.e., (|55 vanishes as I for all n > 2. 

As we noted before, the fact that (|53")) + vanishes with e^ fe ^ is sufficient (exactly as in the lower- 
bound argument of (|80[1 ) to inductively derive the upper-bound (|64[) via the recursive relationship in 
Consequently, the heavy-traffic result (|65|) follows immediately, completing the proof. 
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